[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:25 dist-test-slave-z8x9 krb5kdc[7520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:25 dist-test-slave-z8x9 krb5kdc[7520](info): set up 2 sockets
Aug 12 00:06:25 dist-test-slave-z8x9 krb5kdc[7520](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 00:06:27.280974 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:27 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:27.426252 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:35011
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:35011
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:27.528633 7555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:27.528890 7555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:27.528918 7555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:27.530280 7555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:27.530334 7555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:27.530349 7555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:27.530362 7555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:27.530375 7555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:27.532392 7555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:35011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:35011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:27.532835 7555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:27.533111 7555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:27.540390 7566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:27.540417 7564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:27.540390 7563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:27.540647 7555 server_base.cc:1034] running on GCE node
I20260812 00:06:27.541013 7555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:27.541558 7555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:27.542768 7555 hybrid_clock.cc:648] HybridClock initialized: now 1786493187542747 us; error 97 us; skew 500 ppm
Aug 12 00:06:27 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:27.550271 7555 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:27.551079 7555 webserver.cc:460] Webserver started at http://127.7.85.254:35827/ using document root <none> and password file <none>
I20260812 00:06:27.551384 7555 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:27.551463 7555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:27.552081 7555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:27.553498 7555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "b000f419b51e485b8de83b324640c03d"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "0f83195ac5afa30979547b0e58768f20"
server_key_iv: "1f5493afcb1a696502dca45a5594107e"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.553920 7555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "b000f419b51e485b8de83b324640c03d"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "0f83195ac5afa30979547b0e58768f20"
server_key_iv: "1f5493afcb1a696502dca45a5594107e"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.556108 7555 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:27.557894 7577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:27.558323 7555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:27.558470 7555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "b000f419b51e485b8de83b324640c03d"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "0f83195ac5afa30979547b0e58768f20"
server_key_iv: "1f5493afcb1a696502dca45a5594107e"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.558568 7555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:27.572682 7555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:27.575526 7555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:27.575766 7555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:27.582015 7555 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:35011
I20260812 00:06:27.582168 7685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:35011 every 8 connection(s)
I20260812 00:06:27.582775 7555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:27.583765 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 7555
I20260812 00:06:27.583904 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:27.584818 7686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:27.584921 7511 external_mini_cluster.cc:1428] Setting key 25a93370ef858923537e5124725ca50a
I20260812 00:06:27.589596 7686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d: Bootstrap starting.
I20260812 00:06:27.590874 7686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:27.591393 7686 log.cc:824] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:27.592554 7686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d: No bootstrap required, opened a new log
I20260812 00:06:27.595453 7686 raft_consensus.cc:348] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b000f419b51e485b8de83b324640c03d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35011 } }
I20260812 00:06:27.595659 7686 raft_consensus.cc:374] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:27.595757 7686 raft_consensus.cc:729] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b000f419b51e485b8de83b324640c03d, State: Initialized, Role: FOLLOWER
I20260812 00:06:27.595945 7686 consensus_queue.cc:260] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b000f419b51e485b8de83b324640c03d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35011 } }
I20260812 00:06:27.596129 7686 raft_consensus.cc:388] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:27.596187 7686 raft_consensus.cc:482] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:27.596264 7686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:27.597080 7686 raft_consensus.cc:504] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b000f419b51e485b8de83b324640c03d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35011 } }
I20260812 00:06:27.597441 7686 leader_election.cc:302] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b000f419b51e485b8de83b324640c03d; no voters:
I20260812 00:06:27.597891 7686 leader_election.cc:288] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:27.597970 7694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:27.598145 7694 raft_consensus.cc:686] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [term 1 LEADER]: Becoming Leader. State: Replica: b000f419b51e485b8de83b324640c03d, State: Running, Role: LEADER
I20260812 00:06:27.598294 7694 consensus_queue.cc:237] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b000f419b51e485b8de83b324640c03d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35011 } }
I20260812 00:06:27.598567 7686 sys_catalog.cc:564] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:27.598987 7694 sys_catalog.cc:455] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b000f419b51e485b8de83b324640c03d. Latest consensus state: current_term: 1 leader_uuid: "b000f419b51e485b8de83b324640c03d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b000f419b51e485b8de83b324640c03d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35011 } } }
I20260812 00:06:27.599064 7694 sys_catalog.cc:458] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:27.599416 7706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:27.600396 7696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b000f419b51e485b8de83b324640c03d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b000f419b51e485b8de83b324640c03d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35011 } } }
I20260812 00:06:27.600492 7696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:27.601609 7706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 00:06:27 dist-test-slave-z8x9 krb5kdc[7520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:27.604614 7706 catalog_manager.cc:1347] Generated new cluster ID: 7ead621e2d2446db9d7e67f4094fb228
I20260812 00:06:27.604689 7706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:27.609104 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:27.586644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.7.85.254:35011)
0812 00:06:27.586962 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:27.586974 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:27.587292 (+ 318us) server_negotiation.cc:407] Connection header received
0812 00:06:27.590705 (+ 3413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:27.590713 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:27.590818 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:27.590943 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:27.593499 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:27.594555 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:27.596267 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:27.596871 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:27.596994 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:27.603833 (+ 6839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:27.603838 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:27.603839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:27.603855 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:27.606571 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:27.607208 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:27.607212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:27.607212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:27.607357 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:27.607722 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:27.607724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:27.607724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:27.608096 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:27.608294 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:27.608600 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 00:06:27.608802 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260812 00:06:27.628463 7706 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:27.629406 7706 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:27.633929 7706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b000f419b51e485b8de83b324640c03d: Generated new TSK 0
I20260812 00:06:27.634156 7706 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:27.674028 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:35011
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:27.770283 7737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:27.770588 7737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:27.770637 7737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:27.771874 7737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:27.771927 7737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:27.771955 7737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:27.773597 7737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:35011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:27.773950 7737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:27.774223 7737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:27.781246 7749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:27.781291 7747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:27.781369 7751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:27.781569 7737 server_base.cc:1034] running on GCE node
I20260812 00:06:27.781776 7737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:27.782035 7737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:27.783245 7737 hybrid_clock.cc:648] HybridClock initialized: now 1786493187783219 us; error 41 us; skew 500 ppm
Aug 12 00:06:27 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:27.786013 7737 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:27.786582 7737 webserver.cc:460] Webserver started at http://127.7.85.193:34251/ using document root <none> and password file <none>
I20260812 00:06:27.786831 7737 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:27.786877 7737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:27.787050 7737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:27.788142 7737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "3b14a719ff52425a981820d7f9ad0529"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "c7db146afa12610f327bf3afee77b60f"
server_key_iv: "5da94832b3d0b65f54f9cc599ff5b979"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.788523 7737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "3b14a719ff52425a981820d7f9ad0529"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "c7db146afa12610f327bf3afee77b60f"
server_key_iv: "5da94832b3d0b65f54f9cc599ff5b979"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.790040 7737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:06:27.790860 7764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:27.791019 7737 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:27.791133 7737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "3b14a719ff52425a981820d7f9ad0529"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "c7db146afa12610f327bf3afee77b60f"
server_key_iv: "5da94832b3d0b65f54f9cc599ff5b979"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.791218 7737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:27.799844 7737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:27.802708 7737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:27.802845 7737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:27.803138 7737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:27.803501 7737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:27.803581 7737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:27.803606 7737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:27.803619 7737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:27.812822 7737 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:33259
I20260812 00:06:27.812973 7999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:33259 every 8 connection(s)
I20260812 00:06:27.813338 7737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
Aug 12 00:06:27 dist-test-slave-z8x9 krb5kdc[7520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:27.821760 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 7737
I20260812 00:06:27.821885 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:27.822065 7511 external_mini_cluster.cc:1428] Setting key edf13e40d0384b251851d985c45d9c25
I20260812 00:06:27.824887 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:27.814586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:45937 (local address 127.7.85.254:35011)
0812 00:06:27.814635 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:27.814638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:27.814879 (+ 241us) server_negotiation.cc:407] Connection header received
0812 00:06:27.815346 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:27.815347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:27.815359 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:27.815419 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:27.816684 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:27.817338 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:27.818411 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:27.818624 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:27.818694 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:27.821291 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:27.821293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:27.821293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:27.821301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:27.823141 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:27.823632 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:27.823634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:27.823634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:27.823706 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:27.824053 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:27.824054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:27.824054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:27.824218 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:27.824287 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:27.824660 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 00:06:27.824755 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:06:27.825116 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:27.814548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:35011 (local address 127.7.85.193:45937)
0812 00:06:27.814736 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:27.814782 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:27.815261 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:27.815481 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:27.815485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:27.816350 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:27.816556 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:27.816562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:27.817460 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:27.817461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:27.818309 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:27.818312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:27.818740 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:27.818741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:27.818865 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:27.819548 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:27.819558 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:27.821131 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:27.823287 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:27.823295 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:27.823297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:27.823551 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:27.823808 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:27.823810 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:27.823810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:27.823944 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:27.824313 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:27.824321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:27.824501 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 00:06:27.824627 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:06:27.824825 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260812 00:06:27.825923 8000 heartbeater.cc:347] Connected to a master server at 127.7.85.254:35011
I20260812 00:06:27.826033 8000 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:27.826236 8000 heartbeater.cc:511] Master 127.7.85.254:35011 requested a full tablet report, sending...
I20260812 00:06:27.827549 7617 ts_manager.cc:194] Registered new tserver with Master: 3b14a719ff52425a981820d7f9ad0529 (127.7.85.193:33259)
I20260812 00:06:27.828374 7617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:45937
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:27.858211 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:35011
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:27.951822 8008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:27.952055 8008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:27.952083 8008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:27.953436 8008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:27.953486 8008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:27.953516 8008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:27.955155 8008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:35011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:27.955436 8008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:27.955700 8008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:27.961098 8020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:27.961115 8022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:27.961251 8017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:27.961596 8008 server_base.cc:1034] running on GCE node
I20260812 00:06:27.961831 8008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:27.962101 8008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:27.963299 8008 hybrid_clock.cc:648] HybridClock initialized: now 1786493187963281 us; error 38 us; skew 500 ppm
Aug 12 00:06:27 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:27.966090 8008 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:27.966588 8008 webserver.cc:460] Webserver started at http://127.7.85.194:34547/ using document root <none> and password file <none>
I20260812 00:06:27.966850 8008 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:27.966923 8008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:27.967077 8008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:27.968160 8008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "805dbbc5bb6c43e18b9021154aed0880"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "4816980956f51db33c5040985ce857b2"
server_key_iv: "111e4ef8521227f2a15b6336e814076a"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.968551 8008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "805dbbc5bb6c43e18b9021154aed0880"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "4816980956f51db33c5040985ce857b2"
server_key_iv: "111e4ef8521227f2a15b6336e814076a"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.970130 8008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:27.971051 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:27.971272 8008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:27.971375 8008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "805dbbc5bb6c43e18b9021154aed0880"
format_stamp: "Formatted at 2026-08-12 00:06:27 on dist-test-slave-z8x9"
server_key: "4816980956f51db33c5040985ce857b2"
server_key_iv: "111e4ef8521227f2a15b6336e814076a"
server_key_version: "encryptionkey@0"
I20260812 00:06:27.971465 8008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:27.995759 8008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:27.998493 8008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:27.998683 8008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:27.998960 8008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:27.999342 8008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:27.999398 8008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:27.999444 8008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:27.999511 8008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:28.006911 8008 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:45865
I20260812 00:06:28.006953 8289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:45865 every 8 connection(s)
I20260812 00:06:28.007344 8008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:28.015842 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8008
I20260812 00:06:28.015928 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:28.016108 7511 external_mini_cluster.cc:1428] Setting key 623cb2237cdf3799167a6ab276c27d98
I20260812 00:06:28.019057 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:28.008490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:57787 (local address 127.7.85.254:35011)
0812 00:06:28.008720 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:28.008723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:28.008868 (+ 145us) server_negotiation.cc:407] Connection header received
0812 00:06:28.009522 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:28.009523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:28.009536 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:28.009594 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:28.010645 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.011165 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.012192 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.012412 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.012503 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:28.015331 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:28.015332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:28.015333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:28.015343 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:28.017260 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.017815 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.017816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.017816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.017882 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.018196 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.018197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.018198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.018317 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:28.018384 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:28.018804 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 00:06:28.018924 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260812 00:06:28.019184 8293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:28.008195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:35011 (local address 127.7.85.194:57787)
0812 00:06:28.008490 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:28.008733 (+ 243us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:28.009408 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:28.009677 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:28.009681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:28.010308 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:28.010531 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:28.010536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.011278 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.011279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:28.012093 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:28.012095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.012515 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.012516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:28.012647 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:28.013343 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:28.013358 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:28.015189 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:28.017349 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:28.017357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:28.017360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:28.017711 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:28.017969 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:28.017970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:28.017971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:28.018128 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:28.018396 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:28.018403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:28.018580 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 00:06:28.018702 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 00:06:28.018904 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":89,"threads_started":1}
I20260812 00:06:28.019917 8290 heartbeater.cc:347] Connected to a master server at 127.7.85.254:35011
I20260812 00:06:28.020044 8290 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:28.020246 8290 heartbeater.cc:511] Master 127.7.85.254:35011 requested a full tablet report, sending...
I20260812 00:06:28.020737 7617 ts_manager.cc:194] Registered new tserver with Master: 805dbbc5bb6c43e18b9021154aed0880 (127.7.85.194:45865)
I20260812 00:06:28.021261 7617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:57787
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:28.045428 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:35011
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:28.135553 8296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:28.135782 8296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:28.135808 8296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:28.137068 8296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:28.137110 8296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:28.137151 8296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:28.138705 8296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:35011
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:28.138965 8296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:28.139209 8296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:28.144568 8302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:28.144594 8304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:28.144841 8309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:28.144940 8296 server_base.cc:1034] running on GCE node
I20260812 00:06:28.145162 8296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:28.145464 8296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:28.146634 8296 hybrid_clock.cc:648] HybridClock initialized: now 1786493188146622 us; error 41 us; skew 500 ppm
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493188, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:28.149385 8296 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:28.149895 8296 webserver.cc:460] Webserver started at http://127.7.85.195:36615/ using document root <none> and password file <none>
I20260812 00:06:28.150134 8296 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:28.150202 8296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:28.150352 8296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:28.151500 8296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "85e83471203b4c959a673f260110d167"
format_stamp: "Formatted at 2026-08-12 00:06:28 on dist-test-slave-z8x9"
server_key: "7c9193c2dacdf9fbc8b41a093f38a00f"
server_key_iv: "677063151e50747f804853ddb8555cc2"
server_key_version: "encryptionkey@0"
I20260812 00:06:28.151917 8296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "85e83471203b4c959a673f260110d167"
format_stamp: "Formatted at 2026-08-12 00:06:28 on dist-test-slave-z8x9"
server_key: "7c9193c2dacdf9fbc8b41a093f38a00f"
server_key_iv: "677063151e50747f804853ddb8555cc2"
server_key_version: "encryptionkey@0"
I20260812 00:06:28.153834 8296 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260812 00:06:28.154810 8320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:28.155043 8296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:28.155201 8296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "85e83471203b4c959a673f260110d167"
format_stamp: "Formatted at 2026-08-12 00:06:28 on dist-test-slave-z8x9"
server_key: "7c9193c2dacdf9fbc8b41a093f38a00f"
server_key_iv: "677063151e50747f804853ddb8555cc2"
server_key_version: "encryptionkey@0"
I20260812 00:06:28.155311 8296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:28.163426 8296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:28.165925 8296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:28.166097 8296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:28.166349 8296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:28.166669 8296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:28.166725 8296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:28.166771 8296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:28.166810 8296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:28.174309 8296 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:39613
I20260812 00:06:28.174458 8543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:39613 every 8 connection(s)
I20260812 00:06:28.174805 8296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493188, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:28.181710 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8296
I20260812 00:06:28.181814 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:28.182106 7511 external_mini_cluster.cc:1428] Setting key 56bbb9e8f0e7d3d1e29e302315128a25
I20260812 00:06:28.185400 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:28.175659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:40331 (local address 127.7.85.254:35011)
0812 00:06:28.175728 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:28.175730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:28.175962 (+ 232us) server_negotiation.cc:407] Connection header received
0812 00:06:28.176446 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:28.176447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:28.176459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:28.176520 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:28.177488 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.178029 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.179027 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.179214 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.179281 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:28.181631 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:28.181633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:28.181633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:28.181640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:28.183810 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.184302 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.184304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.184304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.184369 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.184642 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.184643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.184643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.184777 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:28.184845 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:28.185146 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 00:06:28.185303 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 00:06:28.185568 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:28.175641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:35011 (local address 127.7.85.195:40331)
0812 00:06:28.175836 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:28.175874 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:28.176351 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:28.176571 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:28.176574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:28.177202 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:28.177405 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:28.177409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.178119 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.178120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:28.178942 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:28.178945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.179307 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.179308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:28.179417 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:28.179983 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:28.179992 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:28.181482 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:28.183900 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:28.183910 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:28.183913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:28.184214 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:28.184440 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:28.184441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:28.184441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:28.184563 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:28.184847 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:28.184855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:28.185007 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 00:06:28.185123 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 00:06:28.185322 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260812 00:06:28.186124 8547 heartbeater.cc:347] Connected to a master server at 127.7.85.254:35011
I20260812 00:06:28.186221 8547 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:28.186465 8547 heartbeater.cc:511] Master 127.7.85.254:35011 requested a full tablet report, sending...
I20260812 00:06:28.186933 7617 ts_manager.cc:194] Registered new tserver with Master: 85e83471203b4c959a673f260110d167 (127.7.85.195:39613)
I20260812 00:06:28.187395 7617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:40331
I20260812 00:06:28.195791 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493188, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:28.225617 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:28.216320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.7.85.193:33259)
0812 00:06:28.216501 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:28.216506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:28.216515 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:28.216542 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:28.216544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:28.216559 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:28.216645 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:28.217367 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.218105 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.218875 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.219204 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.219295 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:28.221976 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:28.221986 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:28.221987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:28.221996 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:28.223972 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.224513 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.224517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.224517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.224592 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.224904 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.224906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.224906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.225075 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:28.225281 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:28.225442 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 00:06:28.225528 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":71,"threads_started":1}
W20260812 00:06:28.226166 7818 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33202
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[7520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:28.254369 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:28.245497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.7.85.193:33259)
0812 00:06:28.245582 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:28.245584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:28.245598 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:06:28.245680 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:28.245682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:28.245692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:28.245745 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:28.246437 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.246949 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.247749 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:28.247971 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:28.248031 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:28.250508 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:28.250510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:28.250510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:28.250519 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:28.252147 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.253019 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.253020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.253020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.253086 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:28.253662 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:28.253664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:28.253664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:28.253792 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:28.253914 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:28.254097 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 00:06:28.254192 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 00:06:28.255283 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 7737
I20260812 00:06:28.271756 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8008
I20260812 00:06:28.298368 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8296
I20260812 00:06:28.310290 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 7555
2026-08-12T00:06:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2410 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[8570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[8570](info): set up 2 sockets
Aug 12 00:06:28 dist-test-slave-z8x9 krb5kdc[8570](info): commencing operation
krb5kdc: starting...
W20260812 00:06:29.668565 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:29 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:29.816975 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39301
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:39301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:29.911054 8591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:29.911331 8591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:29.911371 8591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:29.912559 8591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:29.912634 8591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:29.912659 8591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:29.912681 8591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:29.912711 8591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:29.914407 8591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:39301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:29.914813 8591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:29.915156 8591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:29.921586 8604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:29.921586 8601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:29.921581 8600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:29.922047 8591 server_base.cc:1034] running on GCE node
I20260812 00:06:29.922353 8591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:29.922660 8591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:29.923888 8591 hybrid_clock.cc:648] HybridClock initialized: now 1786493189923854 us; error 56 us; skew 500 ppm
Aug 12 00:06:29 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493189, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:29.927163 8591 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:29.927677 8591 webserver.cc:460] Webserver started at http://127.7.85.254:38221/ using document root <none> and password file <none>
I20260812 00:06:29.927947 8591 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:29.928020 8591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:29.928174 8591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:29.929333 8591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "970fca876edd4585b458c87ba42bfc6a"
format_stamp: "Formatted at 2026-08-12 00:06:29 on dist-test-slave-z8x9"
server_key: "3ceb03d1cef409f22116c5dca7998aff"
server_key_iv: "361ceb8907520b320016dce18795d379"
server_key_version: "encryptionkey@0"
I20260812 00:06:29.929714 8591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "970fca876edd4585b458c87ba42bfc6a"
format_stamp: "Formatted at 2026-08-12 00:06:29 on dist-test-slave-z8x9"
server_key: "3ceb03d1cef409f22116c5dca7998aff"
server_key_iv: "361ceb8907520b320016dce18795d379"
server_key_version: "encryptionkey@0"
I20260812 00:06:29.931210 8591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:29.932170 8620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:29.932448 8591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:29.932575 8591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "970fca876edd4585b458c87ba42bfc6a"
format_stamp: "Formatted at 2026-08-12 00:06:29 on dist-test-slave-z8x9"
server_key: "3ceb03d1cef409f22116c5dca7998aff"
server_key_iv: "361ceb8907520b320016dce18795d379"
server_key_version: "encryptionkey@0"
I20260812 00:06:29.932664 8591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:29.942590 8591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:29.944963 8591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:29.945144 8591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:29.949245 8591 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:39301
I20260812 00:06:29.949282 8731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:39301 every 8 connection(s)
I20260812 00:06:29.949671 8591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:29.950646 8733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:29.953029 8733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a: Bootstrap starting.
I20260812 00:06:29.953496 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8591
I20260812 00:06:29.953576 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:29.953750 8733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:29.953768 7511 external_mini_cluster.cc:1428] Setting key 16c129fbe4de23d80b3ceff68db3a0d5
I20260812 00:06:29.954104 8733 log.cc:824] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:29.954922 8733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a: No bootstrap required, opened a new log
I20260812 00:06:29.957105 8733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970fca876edd4585b458c87ba42bfc6a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39301 } }
I20260812 00:06:29.957388 8733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:29.957453 8733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970fca876edd4585b458c87ba42bfc6a, State: Initialized, Role: FOLLOWER
I20260812 00:06:29.957625 8733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970fca876edd4585b458c87ba42bfc6a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39301 } }
I20260812 00:06:29.957746 8733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:29.957799 8733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:29.957855 8733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:29.958508 8733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970fca876edd4585b458c87ba42bfc6a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39301 } }
I20260812 00:06:29.958812 8733 leader_election.cc:302] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 970fca876edd4585b458c87ba42bfc6a; no voters:
I20260812 00:06:29.959052 8733 leader_election.cc:288] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:29.959080 8740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:29.959395 8733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:29.959563 8740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [term 1 LEADER]: Becoming Leader. State: Replica: 970fca876edd4585b458c87ba42bfc6a, State: Running, Role: LEADER
I20260812 00:06:29.959749 8740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970fca876edd4585b458c87ba42bfc6a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39301 } }
I20260812 00:06:29.960536 8743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 970fca876edd4585b458c87ba42bfc6a. Latest consensus state: current_term: 1 leader_uuid: "970fca876edd4585b458c87ba42bfc6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970fca876edd4585b458c87ba42bfc6a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39301 } } }
I20260812 00:06:29.960530 8742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "970fca876edd4585b458c87ba42bfc6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "970fca876edd4585b458c87ba42bfc6a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39301 } } }
I20260812 00:06:29.960711 8743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:29.960719 8742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:29.961102 8756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 00:06:29 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:29.961755 8756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:29.963706 8756 catalog_manager.cc:1347] Generated new cluster ID: feb479165fd2422fa0415b8d9621399a
I20260812 00:06:29.963918 8756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:29.966619 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:29.954411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.7.85.254:39301)
0812 00:06:29.954598 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:29.954606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:29.954633 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:06:29.955142 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:29.955149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:29.955210 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:29.955374 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:29.956150 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:29.957065 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:29.958361 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:29.958843 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:29.958964 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:29.962337 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:29.962342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:29.962342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:29.962357 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:29.964800 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:29.965308 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:29.965311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:29.965312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:29.965423 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:29.965660 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:29.965661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:29.965661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:29.965966 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:29.966130 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:29.966250 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 00:06:29.966384 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260812 00:06:29.970144 8756 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:29.970669 8756 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:29.974557 8756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 970fca876edd4585b458c87ba42bfc6a: Generated new TSK 0
I20260812 00:06:29.974779 8756 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:30.014405 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39301
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:30.104312 8778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:30.104535 8778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:30.104560 8778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:30.105770 8778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:30.105820 8778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:30.105849 8778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:30.107448 8778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39301
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:30.107728 8778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:30.107982 8778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:30.113538 8784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:30.113647 8778 server_base.cc:1034] running on GCE node
W20260812 00:06:30.113554 8787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:30.113551 8785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:30.114131 8778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:30.114441 8778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:30.115653 8778 hybrid_clock.cc:648] HybridClock initialized: now 1786493190115638 us; error 30 us; skew 500 ppm
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:30.119341 8778 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:30.119838 8778 webserver.cc:460] Webserver started at http://127.7.85.193:33657/ using document root <none> and password file <none>
I20260812 00:06:30.120088 8778 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:30.120160 8778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:30.120321 8778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:30.121510 8778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "6b43a2762f6a433c9cab19b8aed9b0ac"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "100cfe2ac1e292b9bbeb1be4aa9f3094"
server_key_iv: "5a91dabf2e0ed7b89d8873c5452803f3"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.121866 8778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "6b43a2762f6a433c9cab19b8aed9b0ac"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "100cfe2ac1e292b9bbeb1be4aa9f3094"
server_key_iv: "5a91dabf2e0ed7b89d8873c5452803f3"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.123231 8778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:30.124081 8797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.124337 8778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:30.124449 8778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "6b43a2762f6a433c9cab19b8aed9b0ac"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "100cfe2ac1e292b9bbeb1be4aa9f3094"
server_key_iv: "5a91dabf2e0ed7b89d8873c5452803f3"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.124528 8778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:30.153399 8778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:30.156617 8778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:30.156828 8778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:30.157100 8778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:30.157510 8778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:30.157578 8778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.157622 8778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:30.157665 8778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.164908 8778 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:36351
I20260812 00:06:30.164960 9010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:36351 every 8 connection(s)
I20260812 00:06:30.165405 8778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:30.171460 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8778
I20260812 00:06:30.171581 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:30.171794 7511 external_mini_cluster.cc:1428] Setting key 3a26d400ebc8b89391c131ce80b51abe
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:30.179929 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.166256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:33149 (local address 127.7.85.254:39301)
0812 00:06:30.166308 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.166310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.166518 (+ 208us) server_negotiation.cc:407] Connection header received
0812 00:06:30.167982 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.167984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.167997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.168068 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.169354 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.169903 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.171244 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.171582 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.171679 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.174859 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.174861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.174861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.174870 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.177495 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.178267 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.178270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.178271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.178408 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.178998 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.179000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.179001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.179197 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.179294 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.179742 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.179852 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:06:30.180389 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.166157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39301 (local address 127.7.85.193:33149)
0812 00:06:30.166350 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.166397 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.167764 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.168263 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.168269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.168693 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:30.168983 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.168988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.170057 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.170060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.171087 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.171091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.171732 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.171734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.172098 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.172740 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:30.172750 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:30.174716 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:30.177834 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:30.177845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:30.177848 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:30.178162 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:30.178563 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:30.178565 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:30.178565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:30.178875 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:30.179329 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:30.179337 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:30.179560 (+ 223us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.179706 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.180075 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":113,"threads_started":1}
I20260812 00:06:30.181083 9011 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39301
I20260812 00:06:30.181253 9011 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:30.181496 9011 heartbeater.cc:511] Master 127.7.85.254:39301 requested a full tablet report, sending...
I20260812 00:06:30.182158 8650 ts_manager.cc:194] Registered new tserver with Master: 6b43a2762f6a433c9cab19b8aed9b0ac (127.7.85.193:36351)
I20260812 00:06:30.183144 8650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:33149
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:30.205837 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39301
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:30.299460 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:30.299732 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:30.299762 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:30.301031 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:30.301085 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:30.301115 9018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:30.302801 9018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39301
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:30.303093 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:30.303359 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:30.309461 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:30.309517 9018 server_base.cc:1034] running on GCE node
W20260812 00:06:30.309450 9035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:30.309461 9028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:30.310089 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:30.310331 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:30.311517 9018 hybrid_clock.cc:648] HybridClock initialized: now 1786493190311497 us; error 33 us; skew 500 ppm
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:30.314464 9018 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:30.314976 9018 webserver.cc:460] Webserver started at http://127.7.85.194:42503/ using document root <none> and password file <none>
I20260812 00:06:30.315233 9018 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:30.315304 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:30.315459 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:30.316533 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "39720ba208a749b98e381dde4ed4dfa7"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "55d4903cc8dfdbc5d7e48f5f427a0d55"
server_key_iv: "149663425485831a08c4412e3b3a80b0"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.316896 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "39720ba208a749b98e381dde4ed4dfa7"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "55d4903cc8dfdbc5d7e48f5f427a0d55"
server_key_iv: "149663425485831a08c4412e3b3a80b0"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.318702 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:06:30.319697 9048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.319883 9018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:30.319999 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "39720ba208a749b98e381dde4ed4dfa7"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "55d4903cc8dfdbc5d7e48f5f427a0d55"
server_key_iv: "149663425485831a08c4412e3b3a80b0"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.320083 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:30.334368 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:30.336934 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:30.337098 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:30.337381 9018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:30.337716 9018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:30.337781 9018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.337826 9018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:30.337855 9018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.344046 9018 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35363
I20260812 00:06:30.344096 9255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35363 every 8 connection(s)
I20260812 00:06:30.344419 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:30.353518 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9018
I20260812 00:06:30.353631 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:30.353806 7511 external_mini_cluster.cc:1428] Setting key 7ffeba16e2f5f1effdcea5756850277f
I20260812 00:06:30.354374 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.345142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:33279 (local address 127.7.85.254:39301)
0812 00:06:30.345226 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.345229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.345499 (+ 270us) server_negotiation.cc:407] Connection header received
0812 00:06:30.346465 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.346466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.346478 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.346534 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.347113 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.347639 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.348672 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.348887 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.348954 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.351320 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.351322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.351322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.351329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.352712 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.353121 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.353122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.353122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.353380 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.353641 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.353642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.353643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.353794 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.353884 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.354186 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.354284 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:06:30.354501 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.345127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39301 (local address 127.7.85.194:33279)
0812 00:06:30.345375 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.345413 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.346361 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.346586 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.346589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.346845 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:30.347036 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.347040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.347757 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.347758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.348577 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.348579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.348988 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.348989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.349107 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.349730 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:30.349739 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:30.351212 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:30.352796 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:30.352808 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:30.352810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:30.353040 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:30.353449 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:30.353450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:30.353450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:30.353557 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:30.353875 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:30.353882 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:30.354018 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.354144 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.354291 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":166,"threads_started":1}
I20260812 00:06:30.355010 9257 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39301
I20260812 00:06:30.355119 9257 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:30.355312 9257 heartbeater.cc:511] Master 127.7.85.254:39301 requested a full tablet report, sending...
I20260812 00:06:30.355700 8650 ts_manager.cc:194] Registered new tserver with Master: 39720ba208a749b98e381dde4ed4dfa7 (127.7.85.194:35363)
I20260812 00:06:30.356096 8650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:33279
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:30.386380 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39301
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:40101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:30.476980 9266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:30.477291 9266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:30.477324 9266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:30.478538 9266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:30.478590 9266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:30.478618 9266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:30.480155 9266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:40101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39301
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:30.480423 9266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:30.480691 9266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:30.486044 9286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:30.486013 9283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:30.486117 9266 server_base.cc:1034] running on GCE node
W20260812 00:06:30.486013 9284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:30.486477 9266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:30.486737 9266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:30.487900 9266 hybrid_clock.cc:648] HybridClock initialized: now 1786493190487868 us; error 61 us; skew 500 ppm
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:30.490693 9266 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:30.491108 9266 webserver.cc:460] Webserver started at http://127.7.85.195:39625/ using document root <none> and password file <none>
I20260812 00:06:30.491345 9266 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:30.491393 9266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:30.491528 9266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:30.492630 9266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "515e8bd5b599499ca252e1017a2c613d"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "9272d781b8bb843278c29fcd270b5410"
server_key_iv: "63a2b56ecfd56f58e528a4e7ff02dc52"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.493007 9266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "515e8bd5b599499ca252e1017a2c613d"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "9272d781b8bb843278c29fcd270b5410"
server_key_iv: "63a2b56ecfd56f58e528a4e7ff02dc52"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.494443 9266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:30.495282 9297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.495481 9266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:06:30.495594 9266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "515e8bd5b599499ca252e1017a2c613d"
format_stamp: "Formatted at 2026-08-12 00:06:30 on dist-test-slave-z8x9"
server_key: "9272d781b8bb843278c29fcd270b5410"
server_key_iv: "63a2b56ecfd56f58e528a4e7ff02dc52"
server_key_version: "encryptionkey@0"
I20260812 00:06:30.495697 9266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:30.513829 9266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:30.516790 9266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:30.516988 9266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:30.517392 9266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:30.517740 9266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:30.517800 9266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.517838 9266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:30.517865 9266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:30.524917 9266 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:35177
I20260812 00:06:30.524960 9523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:35177 every 8 connection(s)
I20260812 00:06:30.525352 9266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:30.532955 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9266
I20260812 00:06:30.533089 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:30.533295 7511 external_mini_cluster.cc:1428] Setting key b858fdab9291ae1852e8b5e70d217e3a
I20260812 00:06:30.537276 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.526361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:34189 (local address 127.7.85.254:39301)
0812 00:06:30.526407 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.526409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.526462 (+ 53us) server_negotiation.cc:407] Connection header received
0812 00:06:30.527561 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.527562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.527576 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.527644 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.528424 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.528963 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.530246 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.530513 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.530588 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.533574 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.533576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.533576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.533592 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.535211 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.535902 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.535903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.535904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.535980 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.536326 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.536327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.536328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.536501 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.536560 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.536972 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.537087 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:06:30.537405 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.526163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39301 (local address 127.7.85.195:34189)
0812 00:06:30.526335 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.526381 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.527448 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.527763 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.527771 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.528065 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:30.528312 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.528316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.529110 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.529112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.530119 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.530122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.530665 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.530667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.530789 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.531441 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:30.531451 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:30.533456 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:30.535400 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:30.535409 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:30.535413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:30.535802 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:30.536073 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:30.536074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:30.536074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:30.536235 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:30.536560 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:30.536568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:30.536780 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.536905 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.537103 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260812 00:06:30.537859 9526 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39301
I20260812 00:06:30.537964 9526 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:30.538193 9526 heartbeater.cc:511] Master 127.7.85.254:39301 requested a full tablet report, sending...
I20260812 00:06:30.538657 8650 ts_manager.cc:194] Registered new tserver with Master: 515e8bd5b599499ca252e1017a2c613d (127.7.85.195:35177)
I20260812 00:06:30.539199 8650 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:34189
I20260812 00:06:30.545162 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:06:30.554730 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.547247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.7.85.254:39301)
0812 00:06:30.547322 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.547324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.547405 (+ 81us) server_negotiation.cc:407] Connection header received
0812 00:06:30.547459 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.547460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.547469 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.547522 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.548095 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.548683 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.549488 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.549697 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.549765 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.551473 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.551474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.551474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.551483 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.553127 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.553777 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.553779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.553779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.553852 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.554100 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.554101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.554101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.554290 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.554396 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.554577 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.554659 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:06:30.557811 8650 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:30.560015 8650 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:30.567792 9563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.563912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:35177 (local address 127.0.0.1:42314)
0812 00:06:30.564334 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.564345 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.564397 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.564576 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.564577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.564579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:30.564608 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.564610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.565842 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.565843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.566602 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.566604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.567436 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.567514 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.567529 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.567611 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.567691 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260812 00:06:30.567798 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.564125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.7.85.195:35177)
0812 00:06:30.564377 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.564381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.564392 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:30.564464 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.564465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.564482 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.564582 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:30.564659 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.565735 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.566706 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.567343 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567420 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.567479 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.567644 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.567694 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":133,"threads_started":1}
I20260812 00:06:30.568171 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.563677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35363 (local address 127.0.0.1:38526)
0812 00:06:30.563873 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.563916 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.563992 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.564218 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.564221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.564224 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:30.564311 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.564313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.565536 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.565537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.566786 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.566789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.567900 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.567985 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.567990 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.568042 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.568090 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260812 00:06:30.568274 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.563795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.7.85.193:36351)
0812 00:06:30.564137 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.564141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.564159 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:06:30.564218 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.564220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.564234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.564319 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:30.564536 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.565699 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.566984 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.567701 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567771 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.567824 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.568142 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.568199 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":86,"threads_started":1}
I20260812 00:06:30.568398 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.563678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:36351 (local address 127.0.0.1:36170)
0812 00:06:30.563959 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.563969 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.564012 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.564400 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.564401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.564403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:30.564452 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.564453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.565818 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.565819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.566899 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.566902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.567863 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.567936 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.567940 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.567989 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.568030 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":2,"thread_start_us":107,"threads_started":1}
I20260812 00:06:30.568470 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.563808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.7.85.194:35363)
0812 00:06:30.564025 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.564030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.564043 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:06:30.564088 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.564090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.564108 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.564210 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:30.564405 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.565407 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567053 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.567741 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.567804 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.567860 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.568304 (+ 444us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.568369 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
I20260812 00:06:30.568873 9392 tablet_service.cc:1467] Processing CreateTablet for tablet 11212613b1d34d76a8964e9d542ae197 (DEFAULT_TABLE table=test-table [id=9d7b448a2079406fa2cebba5f040df16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:30.569104 8887 tablet_service.cc:1467] Processing CreateTablet for tablet 11212613b1d34d76a8964e9d542ae197 (DEFAULT_TABLE table=test-table [id=9d7b448a2079406fa2cebba5f040df16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:30.569345 9392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11212613b1d34d76a8964e9d542ae197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:30.569550 8887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11212613b1d34d76a8964e9d542ae197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:30.572072 9568 tablet_bootstrap.cc:492] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d: Bootstrap starting.
I20260812 00:06:30.572712 9577 tablet_bootstrap.cc:492] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: Bootstrap starting.
I20260812 00:06:30.572606 9135 tablet_service.cc:1467] Processing CreateTablet for tablet 11212613b1d34d76a8964e9d542ae197 (DEFAULT_TABLE table=test-table [id=9d7b448a2079406fa2cebba5f040df16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:30.572907 9568 tablet_bootstrap.cc:654] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:30.572959 9135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11212613b1d34d76a8964e9d542ae197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:30.573383 9577 tablet_bootstrap.cc:654] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:30.573701 9568 log.cc:824] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:30.573951 9577 log.cc:824] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:30.574498 9568 tablet_bootstrap.cc:492] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d: No bootstrap required, opened a new log
I20260812 00:06:30.574587 9568 ts_tablet_manager.cc:1397] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:06:30.575979 9577 tablet_bootstrap.cc:492] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: No bootstrap required, opened a new log
I20260812 00:06:30.576061 9577 ts_tablet_manager.cc:1397] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:06:30.576025 9585 tablet_bootstrap.cc:492] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: Bootstrap starting.
I20260812 00:06:30.576364 9568 raft_consensus.cc:348] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.576576 9568 raft_consensus.cc:374] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:30.576611 9568 raft_consensus.cc:729] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515e8bd5b599499ca252e1017a2c613d, State: Initialized, Role: FOLLOWER
I20260812 00:06:30.576732 9568 consensus_queue.cc:260] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.576869 9585 tablet_bootstrap.cc:654] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:30.577046 9568 ts_tablet_manager.cc:1428] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 00:06:30.577327 9526 heartbeater.cc:503] Master 127.7.85.254:39301 was elected leader, sending a full tablet report...
I20260812 00:06:30.577589 9585 log.cc:824] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:30.578011 9577 raft_consensus.cc:348] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.578222 9577 raft_consensus.cc:374] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:30.578274 9577 raft_consensus.cc:729] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b43a2762f6a433c9cab19b8aed9b0ac, State: Initialized, Role: FOLLOWER
I20260812 00:06:30.578317 9585 tablet_bootstrap.cc:492] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: No bootstrap required, opened a new log
I20260812 00:06:30.578392 9585 ts_tablet_manager.cc:1397] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:30.578390 9577 consensus_queue.cc:260] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.578660 9577 ts_tablet_manager.cc:1428] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:06:30.578728 9011 heartbeater.cc:503] Master 127.7.85.254:39301 was elected leader, sending a full tablet report...
I20260812 00:06:30.580787 9585 raft_consensus.cc:348] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.580982 9585 raft_consensus.cc:374] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:30.581033 9585 raft_consensus.cc:729] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39720ba208a749b98e381dde4ed4dfa7, State: Initialized, Role: FOLLOWER
I20260812 00:06:30.581141 9585 consensus_queue.cc:260] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.581466 9585 ts_tablet_manager.cc:1428] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:06:30.581493 9257 heartbeater.cc:503] Master 127.7.85.254:39301 was elected leader, sending a full tablet report...
W20260812 00:06:30.594981 9259 tablet.cc:2369] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:30.624495 9590 raft_consensus.cc:482] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:30.624650 9590 raft_consensus.cc:504] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.625300 9590 leader_election.cc:288] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 515e8bd5b599499ca252e1017a2c613d (127.7.85.195:35177), 6b43a2762f6a433c9cab19b8aed9b0ac (127.7.85.193:36351)
I20260812 00:06:30.628605 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.625497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:36351 (local address 127.7.85.194:49581)
0812 00:06:30.625564 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.625578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.625661 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.625823 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.625823 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.625825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:30.625905 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.625907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.626679 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.626680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.627555 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.627557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.628279 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.628280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.628388 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.628392 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.628456 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.628506 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:06:30.628701 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.625626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:49581 (local address 127.7.85.193:36351)
0812 00:06:30.625677 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.625678 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.625690 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:06:30.625736 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.625737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.625750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.625801 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:30.625989 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.626583 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.627639 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.628172 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.628236 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.628244 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.628554 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.628601 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:06:30.628867 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.625622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:38119 (local address 127.7.85.195:35177)
0812 00:06:30.625678 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.625680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.625852 (+ 172us) server_negotiation.cc:407] Connection header received
0812 00:06:30.625962 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.625964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.625979 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.626034 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:30.626174 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.626693 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.627986 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.628469 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.628534 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.628541 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.628736 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.628786 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 00:06:30.629112 9594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.625497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:35177 (local address 127.7.85.194:38119)
0812 00:06:30.625756 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:30.625765 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:30.625887 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:30.626039 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:30.626040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:30.626043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:30.626106 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.626108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.626814 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.626815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.627861 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:30.627864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.628550 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.628551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:30.628629 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.628634 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:30.628971 (+ 337us) client_negotiation.cc:240] Negotiation successful
0812 00:06:30.629029 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":80,"thread_start_us":107,"threads_started":1}
I20260812 00:06:30.629026 8930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11212613b1d34d76a8964e9d542ae197" candidate_uuid: "39720ba208a749b98e381dde4ed4dfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" is_pre_election: true
I20260812 00:06:30.629292 8930 raft_consensus.cc:2454] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39720ba208a749b98e381dde4ed4dfa7 in term 0.
I20260812 00:06:30.629506 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11212613b1d34d76a8964e9d542ae197" candidate_uuid: "39720ba208a749b98e381dde4ed4dfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515e8bd5b599499ca252e1017a2c613d" is_pre_election: true
I20260812 00:06:30.629624 9055 leader_election.cc:302] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39720ba208a749b98e381dde4ed4dfa7, 6b43a2762f6a433c9cab19b8aed9b0ac; no voters:
I20260812 00:06:30.629693 9433 raft_consensus.cc:2454] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39720ba208a749b98e381dde4ed4dfa7 in term 0.
I20260812 00:06:30.629774 9590 raft_consensus.cc:2781] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:30.629863 9590 raft_consensus.cc:482] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:30.629911 9590 raft_consensus.cc:3037] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:30.630882 9590 raft_consensus.cc:504] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.631160 9590 leader_election.cc:288] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [CANDIDATE]: Term 1 election: Requested vote from peers 515e8bd5b599499ca252e1017a2c613d (127.7.85.195:35177), 6b43a2762f6a433c9cab19b8aed9b0ac (127.7.85.193:36351)
I20260812 00:06:30.631300 9433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11212613b1d34d76a8964e9d542ae197" candidate_uuid: "39720ba208a749b98e381dde4ed4dfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "515e8bd5b599499ca252e1017a2c613d"
I20260812 00:06:30.631578 9433 raft_consensus.cc:3037] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:30.631620 8930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11212613b1d34d76a8964e9d542ae197" candidate_uuid: "39720ba208a749b98e381dde4ed4dfa7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac"
I20260812 00:06:30.631708 8930 raft_consensus.cc:3037] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:30.632562 8930 raft_consensus.cc:2454] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39720ba208a749b98e381dde4ed4dfa7 in term 1.
I20260812 00:06:30.632575 9433 raft_consensus.cc:2454] T 11212613b1d34d76a8964e9d542ae197 P 515e8bd5b599499ca252e1017a2c613d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39720ba208a749b98e381dde4ed4dfa7 in term 1.
I20260812 00:06:30.632769 9055 leader_election.cc:302] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39720ba208a749b98e381dde4ed4dfa7, 6b43a2762f6a433c9cab19b8aed9b0ac; no voters:
I20260812 00:06:30.632879 9590 raft_consensus.cc:2781] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:30.633086 9590 raft_consensus.cc:686] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [term 1 LEADER]: Becoming Leader. State: Replica: 39720ba208a749b98e381dde4ed4dfa7, State: Running, Role: LEADER
I20260812 00:06:30.633224 9590 consensus_queue.cc:237] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } }
I20260812 00:06:30.634372 8645 catalog_manager.cc:5495] T 11212613b1d34d76a8964e9d542ae197 P 39720ba208a749b98e381dde4ed4dfa7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39720ba208a749b98e381dde4ed4dfa7 (127.7.85.194). New cstate: current_term: 1 leader_uuid: "39720ba208a749b98e381dde4ed4dfa7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39720ba208a749b98e381dde4ed4dfa7" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "515e8bd5b599499ca252e1017a2c613d" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 35177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b43a2762f6a433c9cab19b8aed9b0ac" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36351 } health_report { overall_health: UNKNOWN } } }
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:30.650312 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.642175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.7.85.193:36351)
0812 00:06:30.642266 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.642269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.642366 (+ 97us) server_negotiation.cc:407] Connection header received
0812 00:06:30.642463 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.642464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.642476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.642542 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.643144 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.643640 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.644380 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.644587 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.644662 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.647306 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.647343 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.647344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.647352 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.649036 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.649441 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.649444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.649444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.649512 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.649729 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.649730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.649730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.649881 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.650038 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.650115 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.650201 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260812 00:06:30.666239 9012 tablet.cc:2369] T 11212613b1d34d76a8964e9d542ae197 P 6b43a2762f6a433c9cab19b8aed9b0ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:30.681936 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.673290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36188 (local address 127.7.85.193:36351)
0812 00:06:30.673397 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.673400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.673474 (+ 74us) server_negotiation.cc:407] Connection header received
0812 00:06:30.673560 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.673561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.673572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.673629 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.674563 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.675076 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.675971 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.676172 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.676257 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.679039 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.679042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.679042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.679050 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.680549 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.680936 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.680937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.680937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.680992 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.681283 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.681285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.681285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.681411 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.681500 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.681730 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.681807 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[8570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:30.708801 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:30.701075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.7.85.193:36351)
0812 00:06:30.701294 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:30.701297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:30.701401 (+ 104us) server_negotiation.cc:407] Connection header received
0812 00:06:30.701510 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:30.701511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:30.701530 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:30.701597 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:30.702194 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.702664 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.703420 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:30.703608 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:30.703688 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:30.706183 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:30.706184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:30.706185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:30.706192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:30.707546 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.707943 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.707944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.707944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.708003 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:30.708235 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:30.708236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:30.708236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:30.708371 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:30.708459 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:30.708588 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 00:06:30.708654 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
W20260812 00:06:30.709700 8857 tablet_service.cc:2908] Rejecting scan request for tablet 11212613b1d34d76a8964e9d542ae197: Uninitialized: clean time has not yet been initialized
I20260812 00:06:30.711107 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8778
I20260812 00:06:30.742538 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9018
I20260812 00:06:30.768537 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9266
I20260812 00:06:30.784037 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8591
2026-08-12T00:06:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2468 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[9622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[9622](info): set up 2 sockets
Aug 12 00:06:30 dist-test-slave-z8x9 krb5kdc[9622](info): commencing operation
krb5kdc: starting...
W20260812 00:06:32.190037 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:32.337903 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:46777
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:46777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:32.426481 9643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:32.426718 9643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:32.426743 9643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:32.427997 9643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:32.428047 9643 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 00:06:32.428061 9643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:32.428074 9643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:32.428087 9643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:32.428099 9643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:32.429795 9643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:46777
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:46777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:32.430063 9643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:32.430346 9643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:32.435753 9655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:32.435767 9653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:32.435781 9659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:32.436035 9643 server_base.cc:1034] running on GCE node
I20260812 00:06:32.436427 9643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:32.436842 9643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:32.438061 9643 hybrid_clock.cc:648] HybridClock initialized: now 1786493192438046 us; error 69 us; skew 500 ppm
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:32.440985 9643 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:32.441627 9643 webserver.cc:460] Webserver started at http://127.7.85.254:38419/ using document root <none> and password file <none>
I20260812 00:06:32.441903 9643 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:32.441979 9643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:32.442139 9643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:32.443269 9643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "3519f272fb63493db2893d0ecd673731"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "bd0b56a8bfc02ae40acb7a082fab2bef"
server_key_iv: "8ea35908cf02172dd22d989dccccc275"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.443913 9643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "3519f272fb63493db2893d0ecd673731"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "bd0b56a8bfc02ae40acb7a082fab2bef"
server_key_iv: "8ea35908cf02172dd22d989dccccc275"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.445875 9643 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 00:06:32.446866 9667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:32.447156 9643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:32.447324 9643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "3519f272fb63493db2893d0ecd673731"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "bd0b56a8bfc02ae40acb7a082fab2bef"
server_key_iv: "8ea35908cf02172dd22d989dccccc275"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.447429 9643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:32.474094 9643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:32.476701 9643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:32.476893 9643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:32.482370 9643 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:46777
I20260812 00:06:32.482450 9794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:46777 every 8 connection(s)
I20260812 00:06:32.482836 9643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:32.483580 9796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:32.484979 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9643
I20260812 00:06:32.485088 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:32.485280 7511 external_mini_cluster.cc:1428] Setting key 97217c8295ea00ce20e15022058101c5
I20260812 00:06:32.486555 9796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: Bootstrap starting.
I20260812 00:06:32.487187 9796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:32.487541 9796 log.cc:824] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:32.488428 9796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: No bootstrap required, opened a new log
I20260812 00:06:32.490959 9796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3519f272fb63493db2893d0ecd673731" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46777 } }
I20260812 00:06:32.491169 9796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:32.491211 9796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3519f272fb63493db2893d0ecd673731, State: Initialized, Role: FOLLOWER
I20260812 00:06:32.491393 9796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3519f272fb63493db2893d0ecd673731" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46777 } }
I20260812 00:06:32.491472 9796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:32.491495 9796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:32.491564 9796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:32.492141 9796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3519f272fb63493db2893d0ecd673731" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46777 } }
I20260812 00:06:32.492231 9796 leader_election.cc:302] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3519f272fb63493db2893d0ecd673731; no voters:
I20260812 00:06:32.492439 9796 leader_election.cc:288] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:32.492468 9811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:32.492591 9811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [term 1 LEADER]: Becoming Leader. State: Replica: 3519f272fb63493db2893d0ecd673731, State: Running, Role: LEADER
I20260812 00:06:32.492736 9811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3519f272fb63493db2893d0ecd673731" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46777 } }
I20260812 00:06:32.492736 9796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:32.495060 9814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3519f272fb63493db2893d0ecd673731. Latest consensus state: current_term: 1 leader_uuid: "3519f272fb63493db2893d0ecd673731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3519f272fb63493db2893d0ecd673731" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46777 } } }
I20260812 00:06:32.495203 9814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:32.495448 9812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3519f272fb63493db2893d0ecd673731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3519f272fb63493db2893d0ecd673731" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46777 } } }
W20260812 00:06:32.495903 9846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 00:06:32.496280 9812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731 [sys.catalog]: This master's current role is: LEADER
W20260812 00:06:32.496287 9846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:06:32.496381 9847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:32.496572 9847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:32.499712 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.485880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.7.85.254:46777)
0812 00:06:32.486072 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:32.486082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:32.486116 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:06:32.486535 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:32.486541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:32.486588 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:32.486764 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:32.487552 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.488929 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.489792 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.490452 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.490574 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.493614 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:32.493619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:32.493620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:32.493633 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:32.497639 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.498232 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.498236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.498237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.498387 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.498688 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.498690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.498690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.499039 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:32.499218 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:32.499343 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 00:06:32.499500 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260812 00:06:32.501709 9847 catalog_manager.cc:1347] Generated new cluster ID: 8db134586ad245cea1d2ffe7b16c1630
I20260812 00:06:32.501787 9847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:32.505191 9847 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:32.505702 9847 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:32.511518 9847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: Generated new TSK 0
I20260812 00:06:32.511718 9847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:32.553510 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46777
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:32.647984 9856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:32.648239 9856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:32.648272 9856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:32.649777 9856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:32.649829 9856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:32.649855 9856 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:06:32.649919 9856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:32.652064 9856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:46777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:32.652621 9856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:32.653069 9856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:32.659811 9876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:32.659875 9866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:32.660300 9856 server_base.cc:1034] running on GCE node
W20260812 00:06:32.660449 9867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:32.660784 9856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:32.661141 9856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:32.662348 9856 hybrid_clock.cc:648] HybridClock initialized: now 1786493192662323 us; error 41 us; skew 500 ppm
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:32.671030 9856 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:32.671567 9856 webserver.cc:460] Webserver started at http://127.7.85.193:46537/ using document root <none> and password file <none>
I20260812 00:06:32.671815 9856 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:32.671864 9856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:32.672003 9856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:32.673111 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "8ce27fe404a24599af9812f6276a16b3"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "64c925d3ff063bd7cfc673133d7eef35"
server_key_iv: "6255938a90b3de4f2e6e25796d161905"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.673758 9856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "8ce27fe404a24599af9812f6276a16b3"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "64c925d3ff063bd7cfc673133d7eef35"
server_key_iv: "6255938a90b3de4f2e6e25796d161905"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.675836 9856 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:32.677366 9895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:32.677881 9856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 00:06:32.678040 9856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "8ce27fe404a24599af9812f6276a16b3"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "64c925d3ff063bd7cfc673133d7eef35"
server_key_iv: "6255938a90b3de4f2e6e25796d161905"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.678189 9856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:32.707435 9856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:32.710572 9856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:32.710893 9856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:32.711534 9856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:32.711601 9856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:32.711638 9856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:32.711680 9856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:32.719862 9856 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:40293
I20260812 00:06:32.719950 10134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:40293 every 8 connection(s)
I20260812 00:06:32.720315 9856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:32.721067 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9856
I20260812 00:06:32.721210 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:32.721413 7511 external_mini_cluster.cc:1428] Setting key 4ee30ff9d52c11fde5ec59391754c51f
I20260812 00:06:32.724050 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.712411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:44517 (local address 127.7.85.254:46777)
0812 00:06:32.712478 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:32.712481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:32.712683 (+ 202us) server_negotiation.cc:407] Connection header received
0812 00:06:32.713367 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:32.713368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:32.713382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:32.713441 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:32.714093 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.714740 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.716950 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.717250 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.717334 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.719957 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:32.719959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:32.719960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:32.719968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:32.721877 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.722584 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.722585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.722585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.722650 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.722998 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.722999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.723000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.723139 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:32.723224 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:32.723727 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 00:06:32.723900 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 00:06:32.724094 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.712328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.7.85.193:44517)
0812 00:06:32.712553 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:32.712595 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:32.713257 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:32.713511 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:32.713516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:32.713785 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:32.713991 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.713995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.714897 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.714899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.716810 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.716813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.717357 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.717358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.717491 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.718220 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:32.718229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:32.719835 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:32.722061 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.722070 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.722072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.722460 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.722730 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.722731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.722732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.722902 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.723225 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:32.723233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:32.723451 (+ 218us) client_negotiation.cc:769] Sending connection context
0812 00:06:32.723577 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:06:32.723781 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":94,"threads_started":1}
I20260812 00:06:32.725052 10135 heartbeater.cc:347] Connected to a master server at 127.7.85.254:46777
I20260812 00:06:32.725282 10135 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:32.725528 10135 heartbeater.cc:511] Master 127.7.85.254:46777 requested a full tablet report, sending...
I20260812 00:06:32.726408 9703 ts_manager.cc:194] Registered new tserver with Master: 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293)
I20260812 00:06:32.727411 9703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:44517
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:06:32.733489 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.726427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:52777 (local address 127.7.85.254:46777)
0812 00:06:32.726483 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:32.726486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:32.726499 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:06:32.726572 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:32.726573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:32.726586 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:32.726671 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:32.727156 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.728061 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.728918 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.729141 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.729270 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.730312 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:32.730314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:32.730314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:32.730321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:32.732066 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.732462 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.732464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.732464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.732527 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.732764 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.732765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.732766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.732894 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:32.733082 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:32.733284 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 00:06:32.733380 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":28}
I20260812 00:06:32.733498 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.726269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.7.85.193:52777)
0812 00:06:32.726433 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:32.726445 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:32.726496 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:32.726719 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:32.726720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:32.726967 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:32.727041 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.727043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.728200 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.728202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.728823 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.728825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.729270 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.729271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.729338 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.729868 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:32.729873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:32.730198 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:32.732188 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.732189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.732190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.732381 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.732605 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.732606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.732606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.732669 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.733075 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:32.733077 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:32.733136 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 00:06:32.733249 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 00:06:32.733378 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":55,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:32.757679 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46777
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:32.854035 10153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:32.854318 10153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:32.854362 10153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:32.855588 10153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:32.855659 10153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:32.855695 10153 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:06:32.855736 10153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:32.857404 10153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:46777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:32.857707 10153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:32.857965 10153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:32.863500 10167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:32.863495 10163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:32.863489 10162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:32.863571 10153 server_base.cc:1034] running on GCE node
I20260812 00:06:32.863867 10153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:32.864132 10153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:32.865301 10153 hybrid_clock.cc:648] HybridClock initialized: now 1786493192865286 us; error 47 us; skew 500 ppm
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:32.868074 10153 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:32.868566 10153 webserver.cc:460] Webserver started at http://127.7.85.194:33269/ using document root <none> and password file <none>
I20260812 00:06:32.868825 10153 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:32.868898 10153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:32.869059 10153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:32.870375 10153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "469fc47ba0214a98a08e3d7631df2295"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "4a9c1809381b10e637d54f3adf6911e3"
server_key_iv: "6583b51a143596587cffe88a18fcd09e"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.870769 10153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "469fc47ba0214a98a08e3d7631df2295"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "4a9c1809381b10e637d54f3adf6911e3"
server_key_iv: "6583b51a143596587cffe88a18fcd09e"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.872596 10153 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:06:32.873540 10178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:32.873796 10153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:32.873903 10153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "469fc47ba0214a98a08e3d7631df2295"
format_stamp: "Formatted at 2026-08-12 00:06:32 on dist-test-slave-z8x9"
server_key: "4a9c1809381b10e637d54f3adf6911e3"
server_key_iv: "6583b51a143596587cffe88a18fcd09e"
server_key_version: "encryptionkey@0"
I20260812 00:06:32.873988 10153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:32.895232 10153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:32.897794 10153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:32.897984 10153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:32.898665 10153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:32.898721 10153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:32.898758 10153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:32.898782 10153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:06:32 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:32.906939 10153 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:34267
I20260812 00:06:32.907055 10419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:34267 every 8 connection(s)
I20260812 00:06:32.907366 10153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:06:32.910221 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.899237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:60869 (local address 127.7.85.254:46777)
0812 00:06:32.899318 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:32.899321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:32.899565 (+ 244us) server_negotiation.cc:407] Connection header received
0812 00:06:32.900120 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:32.900121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:32.900136 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:32.900260 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:32.900975 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.901883 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.903104 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.903334 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.903422 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.907237 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:32.907241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:32.907241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:32.907253 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:32.908710 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.909161 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.909163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.909163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.909261 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.909539 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.909540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.909540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.909669 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:32.909747 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:32.910042 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 00:06:32.910127 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:06:32.910422 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.899208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.7.85.194:60869)
0812 00:06:32.899417 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:32.899463 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:32.899997 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:32.900386 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:32.900391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:32.900654 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:32.900888 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.900893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.902007 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.902008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.902955 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.902958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.903445 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.903446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.903624 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.904260 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:32.904273 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:32.907138 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:32.908821 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.908829 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.908831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.909076 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.909335 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.909336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.909336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.909459 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.909738 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:32.909745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:32.909894 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 00:06:32.910004 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 00:06:32.910176 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260812 00:06:32.911070 10421 heartbeater.cc:347] Connected to a master server at 127.7.85.254:46777
I20260812 00:06:32.911252 10421 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:32.911490 10421 heartbeater.cc:511] Master 127.7.85.254:46777 requested a full tablet report, sending...
I20260812 00:06:32.912026 9705 ts_manager.cc:194] Registered new tserver with Master: 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267)
I20260812 00:06:32.912640 9705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:60869
I20260812 00:06:32.914942 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10153
I20260812 00:06:32.915055 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:32.915343 7511 external_mini_cluster.cc:1428] Setting key 60b6322312313acc1dff6510f5433bc9
I20260812 00:06:32.918797 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.912143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:46541 (local address 127.7.85.254:46777)
0812 00:06:32.912230 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:32.912232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:32.912290 (+ 58us) server_negotiation.cc:407] Connection header received
0812 00:06:32.912371 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:32.912372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:32.912382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:32.912431 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:32.912763 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.913409 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.914127 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.914339 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.914417 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.915580 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:32.915581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:32.915582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:32.915589 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:32.917359 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.917750 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.917751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.917752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.917813 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:32.918043 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:32.918044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:32.918045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:32.918282 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:32.918333 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:32.918653 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 00:06:32.918735 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:06:32.918788 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:32.912092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.7.85.194:46541)
0812 00:06:32.912236 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:32.912247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:32.912306 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:32.912433 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:32.912434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:32.912630 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:32.912682 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.912684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.913512 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.913513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.914036 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:32.914038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:32.914416 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:32.914417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:32.914493 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:32.915105 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:32.915111 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:32.915495 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:32.917464 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.917465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.917466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.917655 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.917893 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:32.917894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:32.917895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:32.917968 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:32.918389 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:32.918390 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:32.918443 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 00:06:32.918517 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:06:32.918651 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:32.945359 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46777
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:33.038918 10441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:33.039254 10441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:33.039289 10441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:33.040978 10441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:33.041028 10441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:33.041049 10441 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:06:33.041069 10441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:33.043478 10441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:46777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:33.043797 10441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:33.044010 10441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:33.049515 10455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:33.049662 10441 server_base.cc:1034] running on GCE node
W20260812 00:06:33.049707 10465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:33.049741 10454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:33.050041 10441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:33.050326 10441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:33.051584 10441 hybrid_clock.cc:648] HybridClock initialized: now 1786493193051574 us; error 38 us; skew 500 ppm
Aug 12 00:06:33 dist-test-slave-z8x9 krb5kdc[9622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493193, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:33.054479 10441 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:33.055032 10441 webserver.cc:460] Webserver started at http://127.7.85.195:34835/ using document root <none> and password file <none>
I20260812 00:06:33.055372 10441 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:33.055449 10441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:33.055603 10441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:33.056732 10441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "d271563be9ae4dbab4dd866fad59dabb"
format_stamp: "Formatted at 2026-08-12 00:06:33 on dist-test-slave-z8x9"
server_key: "2b1b375dd6eb35d4ef523fffb0ef3296"
server_key_iv: "35cc04234616e48803e35c966410a305"
server_key_version: "encryptionkey@0"
I20260812 00:06:33.057119 10441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "d271563be9ae4dbab4dd866fad59dabb"
format_stamp: "Formatted at 2026-08-12 00:06:33 on dist-test-slave-z8x9"
server_key: "2b1b375dd6eb35d4ef523fffb0ef3296"
server_key_iv: "35cc04234616e48803e35c966410a305"
server_key_version: "encryptionkey@0"
I20260812 00:06:33.058633 10441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:33.059712 10480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:33.060011 10441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:33.060117 10441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "d271563be9ae4dbab4dd866fad59dabb"
format_stamp: "Formatted at 2026-08-12 00:06:33 on dist-test-slave-z8x9"
server_key: "2b1b375dd6eb35d4ef523fffb0ef3296"
server_key_iv: "35cc04234616e48803e35c966410a305"
server_key_version: "encryptionkey@0"
I20260812 00:06:33.060171 10441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:33.075279 10441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:33.078060 10441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:33.078188 10441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:33.078799 10441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:33.078876 10441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:33.078923 10441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:33.078953 10441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:06:33 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493193, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:33.087126 10441 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:41305
I20260812 00:06:33.087304 10705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:41305 every 8 connection(s)
I20260812 00:06:33.087580 10441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:33.090703 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.079833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:42825 (local address 127.7.85.254:46777)
0812 00:06:33.079922 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.079925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.080209 (+ 284us) server_negotiation.cc:407] Connection header received
0812 00:06:33.081349 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.081352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.081372 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.081534 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:33.082691 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.083211 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.084289 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.084496 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.084576 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.087040 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:33.087042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:33.087042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:33.087050 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:33.088717 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.089573 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.089574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.089575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.089641 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.089959 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.089960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.089961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.090090 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:33.090144 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.090527 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.090621 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 00:06:33.090762 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.079677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.7.85.195:42825)
0812 00:06:33.079995 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.080066 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.081132 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.081737 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.081744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.082231 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:33.082572 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.082581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.083326 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.083328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.084163 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.084165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.084599 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.084600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.084768 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.085419 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:33.085438 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:33.086930 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:33.088861 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:33.088874 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:33.088877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:33.089449 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:33.089735 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:33.089737 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:33.089737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:33.089868 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:33.090203 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:33.090209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:33.090352 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.090442 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.090585 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":113,"threads_started":1}
I20260812 00:06:33.091282 10707 heartbeater.cc:347] Connected to a master server at 127.7.85.254:46777
I20260812 00:06:33.091374 10707 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:33.091567 10707 heartbeater.cc:511] Master 127.7.85.254:46777 requested a full tablet report, sending...
I20260812 00:06:33.091979 9703 ts_manager.cc:194] Registered new tserver with Master: d271563be9ae4dbab4dd866fad59dabb (127.7.85.195:41305)
I20260812 00:06:33.092257 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10441
I20260812 00:06:33.092355 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:33.092510 9703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:42825
I20260812 00:06:33.092543 7511 external_mini_cluster.cc:1428] Setting key 01311d77fcc11ffec57815d59ac518bc
I20260812 00:06:33.094033 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:06:33.100070 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.092497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.7.85.195:36533)
0812 00:06:33.092721 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.092733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.092793 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.092949 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.092950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.093196 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:33.093261 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.093264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.094032 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.094033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.094848 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.094851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.095225 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.095226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.095313 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.095878 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:33.095884 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:33.096320 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:33.098194 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:33.098205 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:33.098206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:33.098864 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:33.099192 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:33.099193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:33.099193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:33.099298 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:33.099665 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:33.099666 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:33.099730 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.099830 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.099964 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
I20260812 00:06:33.100067 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.092542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:36533 (local address 127.7.85.254:46777)
0812 00:06:33.092632 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.092633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.092779 (+ 146us) server_negotiation.cc:407] Connection header received
0812 00:06:33.092870 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.092871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.092883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.092934 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:33.093348 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.093918 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.094958 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.095142 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.095210 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.096439 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:33.096441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:33.096442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:33.096450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:33.098037 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.098968 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.098969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.098970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.099053 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.099377 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.099379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.099379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.099552 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:33.099632 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.099877 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.099961 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 00:06:33 dist-test-slave-z8x9 krb5kdc[9622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493193, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:06:33 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493193, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:33.124230 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.116022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35744 (local address 127.7.85.254:46777)
0812 00:06:33.116132 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.116133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.116144 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:33.116171 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.116172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.116183 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.116305 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:33.117001 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.117608 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.118378 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.118565 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.118644 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.121365 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:33.121367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:33.121367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:33.121383 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:33.122869 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.123360 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.123362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.123362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.123427 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.123634 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.123636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.123636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.123763 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:33.123879 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.124022 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.124109 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 00:06:33.126389 9703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:33.128013 9703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:33.135530 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.130696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:41305 (local address 127.0.0.1:39868)
0812 00:06:33.131134 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.131179 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.131270 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.131575 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.131576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.131578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.131686 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.131688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.132974 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.132975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.134458 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.134461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.135200 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.135200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.135289 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.135304 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.135380 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.135439 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":107,"thread_start_us":81,"threads_started":1}
I20260812 00:06:33.135763 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.131186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.7.85.194:34267)
0812 00:06:33.131458 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.131463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.131711 (+ 248us) server_negotiation.cc:407] Connection header received
0812 00:06:33.131757 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.131759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.131775 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.131866 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.131974 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.132958 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.134553 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.135284 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.135352 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.135400 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.135488 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.135533 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":89,"threads_started":1}
I20260812 00:06:33.135821 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.130925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.7.85.195:41305)
0812 00:06:33.131118 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.131124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.131382 (+ 258us) server_negotiation.cc:407] Connection header received
0812 00:06:33.131424 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.131426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.131443 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.131513 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.131782 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.132866 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.134566 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.135098 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.135200 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.135264 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.135690 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.135733 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260812 00:06:33.135926 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.130929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.7.85.193:40293)
0812 00:06:33.131209 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.131213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.131307 (+ 94us) server_negotiation.cc:407] Connection header received
0812 00:06:33.131387 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.131390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.131407 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.131486 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.131715 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.132866 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.134424 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.135198 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.135292 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.135356 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.135602 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.135645 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":110,"threads_started":1}
I20260812 00:06:33.136266 10746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.130925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:34267 (local address 127.0.0.1:42234)
0812 00:06:33.131590 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.131600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.131651 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.131863 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.131863 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.131865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.131916 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.131917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.133037 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.133038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.134459 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.134461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.135367 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.135368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.135422 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.135425 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.135476 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.135511 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":166,"thread_start_us":97,"threads_started":1}
I20260812 00:06:33.136407 10755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.130773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40293 (local address 127.0.0.1:35346)
0812 00:06:33.131219 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.131231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.131301 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.131568 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.131572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.131573 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.131651 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.131653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.132985 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.132986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.134316 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.134319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.135394 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.135396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.135461 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.135466 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.135517 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.135559 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":2,"thread_start_us":126,"threads_started":1}
I20260812 00:06:33.136502 10595 tablet_service.cc:1467] Processing CreateTablet for tablet dbfe8ec3849d410cb89d1b244cd1df8e (DEFAULT_TABLE table=test-table [id=87d7a7369ccd494a82ea82e5ecd1e87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:33.136958 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfe8ec3849d410cb89d1b244cd1df8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.136996 10285 tablet_service.cc:1467] Processing CreateTablet for tablet dbfe8ec3849d410cb89d1b244cd1df8e (DEFAULT_TABLE table=test-table [id=87d7a7369ccd494a82ea82e5ecd1e87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:33.137105 9988 tablet_service.cc:1467] Processing CreateTablet for tablet dbfe8ec3849d410cb89d1b244cd1df8e (DEFAULT_TABLE table=test-table [id=87d7a7369ccd494a82ea82e5ecd1e87d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:33.137490 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfe8ec3849d410cb89d1b244cd1df8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.137544 9988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbfe8ec3849d410cb89d1b244cd1df8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.140398 10764 tablet_bootstrap.cc:492] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Bootstrap starting.
I20260812 00:06:33.141297 10764 tablet_bootstrap.cc:654] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.141472 10768 tablet_bootstrap.cc:492] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Bootstrap starting.
I20260812 00:06:33.141675 10764 log.cc:824] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:33.141644 10767 tablet_bootstrap.cc:492] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Bootstrap starting.
I20260812 00:06:33.142122 10768 tablet_bootstrap.cc:654] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.142475 10768 log.cc:824] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:33.142678 10764 tablet_bootstrap.cc:492] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: No bootstrap required, opened a new log
I20260812 00:06:33.142781 10764 ts_tablet_manager.cc:1397] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 00:06:33.142812 10767 tablet_bootstrap.cc:654] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.143123 10767 log.cc:824] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:33.143267 10768 tablet_bootstrap.cc:492] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: No bootstrap required, opened a new log
I20260812 00:06:33.143343 10768 ts_tablet_manager.cc:1397] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:06:33.143846 10767 tablet_bootstrap.cc:492] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: No bootstrap required, opened a new log
I20260812 00:06:33.144047 10767 ts_tablet_manager.cc:1397] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:33.144829 10764 raft_consensus.cc:348] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.145015 10764 raft_consensus.cc:374] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.145059 10764 raft_consensus.cc:729] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469fc47ba0214a98a08e3d7631df2295, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.145228 10764 consensus_queue.cc:260] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.145362 10768 raft_consensus.cc:348] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.145521 10768 raft_consensus.cc:374] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.145546 10764 ts_tablet_manager.cc:1428] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 00:06:33.145565 10768 raft_consensus.cc:729] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce27fe404a24599af9812f6276a16b3, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.145692 10421 heartbeater.cc:503] Master 127.7.85.254:46777 was elected leader, sending a full tablet report...
I20260812 00:06:33.145720 10768 consensus_queue.cc:260] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.145871 10767 raft_consensus.cc:348] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.145985 10768 ts_tablet_manager.cc:1428] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 00:06:33.146006 10135 heartbeater.cc:503] Master 127.7.85.254:46777 was elected leader, sending a full tablet report...
I20260812 00:06:33.146023 10767 raft_consensus.cc:374] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.146066 10767 raft_consensus.cc:729] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d271563be9ae4dbab4dd866fad59dabb, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.146171 10767 consensus_queue.cc:260] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.146448 10767 ts_tablet_manager.cc:1428] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 00:06:33.146600 10707 heartbeater.cc:503] Master 127.7.85.254:46777 was elected leader, sending a full tablet report...
W20260812 00:06:33.158141 10425 tablet.cc:2369] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:06:33.221309 10138 tablet.cc:2369] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:33.311013 10777 raft_consensus.cc:482] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:33.311170 10777 raft_consensus.cc:504] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.311734 10777 leader_election.cc:288] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267), 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293)
I20260812 00:06:33.315841 10779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.312021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:34267 (local address 127.7.85.195:51741)
0812 00:06:33.312313 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.312325 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.312402 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.312803 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.312804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.312808 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.312965 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.312967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.313993 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.313995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.314949 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.314952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.315516 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.315517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.315608 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.315612 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.315686 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.315730 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":14,"thread_start_us":399,"threads_started":1}
I20260812 00:06:33.315847 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.312128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:51741 (local address 127.7.85.194:34267)
0812 00:06:33.312233 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.312235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.312376 (+ 141us) server_negotiation.cc:407] Connection header received
0812 00:06:33.312652 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.312654 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.312679 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.312786 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.313055 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.313880 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.315044 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.315421 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.315516 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.315525 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.315705 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.315748 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 00:06:33.316012 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.312019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40293 (local address 127.7.85.195:35725)
0812 00:06:33.312091 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.312104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.312157 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.312461 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.312463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.312465 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.312561 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.312564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.313459 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.313461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.314428 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.314431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.315726 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.315727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.315811 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.315814 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.315890 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.315940 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 00:06:33.316020 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.312129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:35725 (local address 127.7.85.193:40293)
0812 00:06:33.312195 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.312197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.312208 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:33.312245 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.312246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.312261 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.312357 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.312771 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.313346 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.314855 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.315628 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.315733 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.315744 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.315913 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.315956 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 00:06:33.316411 10026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe8ec3849d410cb89d1b244cd1df8e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce27fe404a24599af9812f6276a16b3" is_pre_election: true
I20260812 00:06:33.316433 10334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe8ec3849d410cb89d1b244cd1df8e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469fc47ba0214a98a08e3d7631df2295" is_pre_election: true
I20260812 00:06:33.316587 10026 raft_consensus.cc:2454] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 0.
I20260812 00:06:33.316598 10334 raft_consensus.cc:2454] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 0.
I20260812 00:06:33.316951 10482 leader_election.cc:302] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469fc47ba0214a98a08e3d7631df2295, d271563be9ae4dbab4dd866fad59dabb; no voters:
I20260812 00:06:33.317238 10777 raft_consensus.cc:2781] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:33.317301 10777 raft_consensus.cc:482] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:33.317330 10777 raft_consensus.cc:3037] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.318768 10777 raft_consensus.cc:504] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.318936 10777 leader_election.cc:288] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 election: Requested vote from peers 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267), 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293)
I20260812 00:06:33.319135 10026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe8ec3849d410cb89d1b244cd1df8e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce27fe404a24599af9812f6276a16b3"
I20260812 00:06:33.319185 10334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbfe8ec3849d410cb89d1b244cd1df8e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469fc47ba0214a98a08e3d7631df2295"
I20260812 00:06:33.319262 10026 raft_consensus.cc:3037] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.319276 10334 raft_consensus.cc:3037] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.320080 10026 raft_consensus.cc:2454] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 1.
I20260812 00:06:33.320112 10334 raft_consensus.cc:2454] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 1.
I20260812 00:06:33.320370 10484 leader_election.cc:302] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ce27fe404a24599af9812f6276a16b3, d271563be9ae4dbab4dd866fad59dabb; no voters:
I20260812 00:06:33.320502 10777 raft_consensus.cc:2781] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:33.320746 10777 raft_consensus.cc:686] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 1 LEADER]: Becoming Leader. State: Replica: d271563be9ae4dbab4dd866fad59dabb, State: Running, Role: LEADER
I20260812 00:06:33.320895 10777 consensus_queue.cc:237] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } }
I20260812 00:06:33.322190 9705 catalog_manager.cc:5495] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb reported cstate change: term changed from 0 to 1, leader changed from <none> to d271563be9ae4dbab4dd866fad59dabb (127.7.85.195). New cstate: current_term: 1 leader_uuid: "d271563be9ae4dbab4dd866fad59dabb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } health_report { overall_health: UNKNOWN } } }
W20260812 00:06:33.338583 10708 tablet.cc:2369] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:33.373920 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.370964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39876 (local address 127.7.85.195:41305)
0812 00:06:33.371062 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.371064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.371092 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:06:33.371243 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.371244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.371275 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.371353 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:33.371527 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.372019 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.372852 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.373050 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.373128 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.373477 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.373606 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.373715 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.373772 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.373800 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:06:33.377291 10026 raft_consensus.cc:1261] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Refusing update from remote peer d271563be9ae4dbab4dd866fad59dabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:33.377291 10334 raft_consensus.cc:1261] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Refusing update from remote peer d271563be9ae4dbab4dd866fad59dabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:33.378058 10782 consensus_queue.cc:1035] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:33.378374 10777 consensus_queue.cc:1035] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:33.381621 10787 mvcc.cc:205] Tried to move back new op lower bound from 7317476120067596288 to 7317476119844057088. Current Snapshot: MvccSnapshot[applied={T|T < 7317476120067596288}]
I20260812 00:06:33.383733 10788 mvcc.cc:205] Tried to move back new op lower bound from 7317476120067596288 to 7317476119844057088. Current Snapshot: MvccSnapshot[applied={T|T < 7317476120067596288}]
I20260812 00:06:33.383126 10789 mvcc.cc:205] Tried to move back new op lower bound from 7317476120067596288 to 7317476119844057088. Current Snapshot: MvccSnapshot[applied={T|T < 7317476120067596288 or (T in {7317476120067596288})}]
I20260812 00:06:33.389241 9703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:33.389416 9703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:33.390597 9703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: Sending DeleteTablet for 3 replicas of tablet dbfe8ec3849d410cb89d1b244cd1df8e
I20260812 00:06:33.391263 10285 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfe8ec3849d410cb89d1b244cd1df8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:33 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:42234
I20260812 00:06:33.391397 9703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:33.391651 9703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:33.392010 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfe8ec3849d410cb89d1b244cd1df8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:33 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39868
I20260812 00:06:33.392388 10796 tablet_replica.cc:330] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: stopping tablet replica
I20260812 00:06:33.392566 10796 raft_consensus.cc:2229] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:33.392732 10796 raft_consensus.cc:2258] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:33.392954 10798 tablet_replica.cc:330] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: stopping tablet replica
I20260812 00:06:33.393072 10798 raft_consensus.cc:2229] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:33.393308 10798 raft_consensus.cc:2258] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:33.393433 9988 tablet_service.cc:1514] Processing DeleteTablet for tablet dbfe8ec3849d410cb89d1b244cd1df8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:33 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:35346
I20260812 00:06:33.393975 10796 ts_tablet_manager.cc:1905] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:33.394097 10595 tablet_service.cc:1467] Processing CreateTablet for tablet e28905461eb44a99927cd07e226c320e (DEFAULT_TABLE table=test-table [id=c13d68cf4f08466c8c9a652b30371dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:33.394290 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28905461eb44a99927cd07e226c320e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.395582 10767 tablet_bootstrap.cc:492] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Bootstrap starting.
I20260812 00:06:33.395797 10285 tablet_service.cc:1467] Processing CreateTablet for tablet e28905461eb44a99927cd07e226c320e (DEFAULT_TABLE table=test-table [id=c13d68cf4f08466c8c9a652b30371dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:33.395929 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28905461eb44a99927cd07e226c320e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.396056 10767 tablet_bootstrap.cc:654] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.396278 9988 tablet_service.cc:1467] Processing CreateTablet for tablet e28905461eb44a99927cd07e226c320e (DEFAULT_TABLE table=test-table [id=c13d68cf4f08466c8c9a652b30371dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:33.396420 9988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28905461eb44a99927cd07e226c320e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.396818 10796 ts_tablet_manager.cc:1918] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:33.396888 10796 log.cc:1196] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/dbfe8ec3849d410cb89d1b244cd1df8e
I20260812 00:06:33.397235 10796 ts_tablet_manager.cc:1939] T dbfe8ec3849d410cb89d1b244cd1df8e P 469fc47ba0214a98a08e3d7631df2295: Deleting consensus metadata
I20260812 00:06:33.397552 10764 tablet_bootstrap.cc:492] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Bootstrap starting.
I20260812 00:06:33.398008 10764 tablet_bootstrap.cc:654] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.397984 9668 catalog_manager.cc:4841] TS 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267): tablet dbfe8ec3849d410cb89d1b244cd1df8e (table test-table [id=87d7a7369ccd494a82ea82e5ecd1e87d]) successfully deleted
I20260812 00:06:33.398252 10767 tablet_bootstrap.cc:492] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: No bootstrap required, opened a new log
I20260812 00:06:33.398311 10767 ts_tablet_manager.cc:1397] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 00:06:33.398514 10767 raft_consensus.cc:348] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.398628 10767 raft_consensus.cc:374] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.398656 10767 raft_consensus.cc:729] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d271563be9ae4dbab4dd866fad59dabb, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.398712 10767 consensus_queue.cc:260] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.398756 10764 tablet_bootstrap.cc:492] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: No bootstrap required, opened a new log
I20260812 00:06:33.398799 10764 ts_tablet_manager.cc:1397] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 00:06:33.398825 10767 ts_tablet_manager.cc:1428] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 00:06:33.398959 10764 raft_consensus.cc:348] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.399039 10764 raft_consensus.cc:374] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.399070 10764 raft_consensus.cc:729] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469fc47ba0214a98a08e3d7631df2295, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.399117 10764 consensus_queue.cc:260] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.399233 10764 ts_tablet_manager.cc:1428] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:33.399489 10768 tablet_bootstrap.cc:492] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Bootstrap starting.
I20260812 00:06:33.399644 10804 tablet_replica.cc:330] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: stopping tablet replica
I20260812 00:06:33.399778 10804 raft_consensus.cc:2229] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:33.399910 10804 raft_consensus.cc:2258] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:33.399926 10768 tablet_bootstrap.cc:654] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.400300 10798 ts_tablet_manager.cc:1905] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:33.400435 10804 ts_tablet_manager.cc:1905] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:33.400708 10768 tablet_bootstrap.cc:492] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: No bootstrap required, opened a new log
I20260812 00:06:33.400758 10768 ts_tablet_manager.cc:1397] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:33.400913 10768 raft_consensus.cc:348] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.400972 10768 raft_consensus.cc:374] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.400993 10768 raft_consensus.cc:729] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce27fe404a24599af9812f6276a16b3, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.401034 10768 consensus_queue.cc:260] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.401134 10768 ts_tablet_manager.cc:1428] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:33.402653 10804 ts_tablet_manager.cc:1918] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:33.402685 10798 ts_tablet_manager.cc:1918] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:33.402872 10804 log.cc:1196] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/dbfe8ec3849d410cb89d1b244cd1df8e
I20260812 00:06:33.402874 10798 log.cc:1196] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/dbfe8ec3849d410cb89d1b244cd1df8e
I20260812 00:06:33.403376 10798 ts_tablet_manager.cc:1939] T dbfe8ec3849d410cb89d1b244cd1df8e P d271563be9ae4dbab4dd866fad59dabb: Deleting consensus metadata
I20260812 00:06:33.403612 10804 ts_tablet_manager.cc:1939] T dbfe8ec3849d410cb89d1b244cd1df8e P 8ce27fe404a24599af9812f6276a16b3: Deleting consensus metadata
I20260812 00:06:33.403901 9668 catalog_manager.cc:4841] TS d271563be9ae4dbab4dd866fad59dabb (127.7.85.195:41305): tablet dbfe8ec3849d410cb89d1b244cd1df8e (table test-table [id=87d7a7369ccd494a82ea82e5ecd1e87d]) successfully deleted
I20260812 00:06:33.404079 9670 catalog_manager.cc:4841] TS 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293): tablet dbfe8ec3849d410cb89d1b244cd1df8e (table test-table [id=87d7a7369ccd494a82ea82e5ecd1e87d]) successfully deleted
I20260812 00:06:33.562583 10777 raft_consensus.cc:482] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:33.562716 10777 raft_consensus.cc:504] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.562980 10777 leader_election.cc:288] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293), 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267)
I20260812 00:06:33.563237 10026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28905461eb44a99927cd07e226c320e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce27fe404a24599af9812f6276a16b3" is_pre_election: true
I20260812 00:06:33.563264 10334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28905461eb44a99927cd07e226c320e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469fc47ba0214a98a08e3d7631df2295" is_pre_election: true
I20260812 00:06:33.563376 10026 raft_consensus.cc:2454] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 0.
I20260812 00:06:33.563390 10334 raft_consensus.cc:2454] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 0.
I20260812 00:06:33.563547 10482 leader_election.cc:302] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469fc47ba0214a98a08e3d7631df2295, d271563be9ae4dbab4dd866fad59dabb; no voters:
I20260812 00:06:33.563694 10777 raft_consensus.cc:2781] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:33.563755 10777 raft_consensus.cc:482] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:33.563771 10777 raft_consensus.cc:3037] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.564530 10777 raft_consensus.cc:504] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.564733 10777 leader_election.cc:288] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 election: Requested vote from peers 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293), 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267)
I20260812 00:06:33.564844 10026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28905461eb44a99927cd07e226c320e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ce27fe404a24599af9812f6276a16b3"
I20260812 00:06:33.564939 10026 raft_consensus.cc:3037] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.565001 10334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28905461eb44a99927cd07e226c320e" candidate_uuid: "d271563be9ae4dbab4dd866fad59dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469fc47ba0214a98a08e3d7631df2295"
I20260812 00:06:33.565121 10334 raft_consensus.cc:3037] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.565740 10026 raft_consensus.cc:2454] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 1.
I20260812 00:06:33.565827 10334 raft_consensus.cc:2454] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d271563be9ae4dbab4dd866fad59dabb in term 1.
I20260812 00:06:33.565948 10484 leader_election.cc:302] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ce27fe404a24599af9812f6276a16b3, d271563be9ae4dbab4dd866fad59dabb; no voters:
I20260812 00:06:33.566048 10777 raft_consensus.cc:2781] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:33.566100 10777 raft_consensus.cc:686] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 1 LEADER]: Becoming Leader. State: Replica: d271563be9ae4dbab4dd866fad59dabb, State: Running, Role: LEADER
I20260812 00:06:33.566160 10777 consensus_queue.cc:237] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.566869 9705 catalog_manager.cc:5495] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb reported cstate change: term changed from 0 to 1, leader changed from <none> to d271563be9ae4dbab4dd866fad59dabb (127.7.85.195). New cstate: current_term: 1 leader_uuid: "d271563be9ae4dbab4dd866fad59dabb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } health_report { overall_health: HEALTHY } } }
Aug 12 00:06:33 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:33.626689 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.618521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46777 (local address 127.0.0.1:35756)
0812 00:06:33.618734 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.618745 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.618822 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.619011 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.619012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.619369 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:33.619416 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.619418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.620124 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.620125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.620669 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.620671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.621023 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.621024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.621069 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.621800 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:33.621805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:33.623280 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:33.625336 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:33.625345 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:33.625348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:33.625637 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:33.625925 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:33.625926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:33.625927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:33.626061 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:33.626379 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:33.626387 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:33.626436 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.626495 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.626605 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
I20260812 00:06:33.626752 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.618659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.7.85.254:46777)
0812 00:06:33.618741 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.618743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.618878 (+ 135us) server_negotiation.cc:407] Connection header received
0812 00:06:33.618913 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.618914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.618928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.618989 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:33.619506 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.620004 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.620756 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.620944 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.621003 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.623387 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:33.623388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:33.623388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:33.623395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:33.625150 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.625762 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.625763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.625764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.625833 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.626161 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.626163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.626163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.626302 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:33.626374 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.626547 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.626663 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":55}
I20260812 00:06:33.629349 9705 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:35756:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 00:06:33.629670 9705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:33.631505 10285 tablet_service.cc:1467] Processing CreateTablet for tablet a00de0debd0343e18f1b38fb7c165c30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b64211f5f1a747a08e144d9609b2e693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:06:33.631515 10595 tablet_service.cc:1467] Processing CreateTablet for tablet a00de0debd0343e18f1b38fb7c165c30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b64211f5f1a747a08e144d9609b2e693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:06:33.631567 9988 tablet_service.cc:1467] Processing CreateTablet for tablet a00de0debd0343e18f1b38fb7c165c30 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b64211f5f1a747a08e144d9609b2e693]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:06:33.631696 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a00de0debd0343e18f1b38fb7c165c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.631695 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a00de0debd0343e18f1b38fb7c165c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.631825 9988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a00de0debd0343e18f1b38fb7c165c30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:33.633459 10767 tablet_bootstrap.cc:492] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: Bootstrap starting.
I20260812 00:06:33.633518 10768 tablet_bootstrap.cc:492] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: Bootstrap starting.
I20260812 00:06:33.633936 10767 tablet_bootstrap.cc:654] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.633967 10768 tablet_bootstrap.cc:654] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.634684 10768 tablet_bootstrap.cc:492] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: No bootstrap required, opened a new log
I20260812 00:06:33.634737 10767 tablet_bootstrap.cc:492] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: No bootstrap required, opened a new log
I20260812 00:06:33.634756 10768 ts_tablet_manager.cc:1397] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:33.634778 10767 ts_tablet_manager.cc:1397] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:33.634900 10767 raft_consensus.cc:348] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.634966 10767 raft_consensus.cc:374] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.634980 10767 raft_consensus.cc:729] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d271563be9ae4dbab4dd866fad59dabb, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.635013 10767 consensus_queue.cc:260] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.635075 10767 ts_tablet_manager.cc:1428] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:33.635175 10777 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.635167 10768 raft_consensus.cc:348] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.635350 10768 raft_consensus.cc:374] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.635378 10768 raft_consensus.cc:729] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce27fe404a24599af9812f6276a16b3, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.635356 10777 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.635566 10764 tablet_bootstrap.cc:492] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: Bootstrap starting.
I20260812 00:06:33.635572 10768 consensus_queue.cc:260] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.635831 10768 ts_tablet_manager.cc:1428] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:33.636116 10764 tablet_bootstrap.cc:654] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:33.636106 10773 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.636200 10773 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.636840 10764 tablet_bootstrap.cc:492] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: No bootstrap required, opened a new log
I20260812 00:06:33.636904 10764 ts_tablet_manager.cc:1397] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 00:06:33.637058 10764 raft_consensus.cc:348] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.637151 10764 raft_consensus.cc:374] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:33.637216 10764 raft_consensus.cc:729] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469fc47ba0214a98a08e3d7631df2295, State: Initialized, Role: FOLLOWER
I20260812 00:06:33.637262 10764 consensus_queue.cc:260] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.637365 10764 ts_tablet_manager.cc:1428] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:06:33.637420 10772 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.637494 10772 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.729346 10773 raft_consensus.cc:482] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:33.729467 10773 raft_consensus.cc:504] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.729815 10773 leader_election.cc:288] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267), d271563be9ae4dbab4dd866fad59dabb (127.7.85.195:41305)
I20260812 00:06:33.733443 10826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.730067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:34267 (local address 127.7.85.193:56619)
0812 00:06:33.730268 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.730281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.730348 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.730587 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.730587 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.730592 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.730661 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.730664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.731401 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.731402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.732575 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.732578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.733130 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.733131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.733228 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.733240 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.733308 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.733350 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":128,"threads_started":1}
I20260812 00:06:33.733431 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.730183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:56619 (local address 127.7.85.194:34267)
0812 00:06:33.730259 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.730262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.730335 (+ 73us) server_negotiation.cc:407] Connection header received
0812 00:06:33.730452 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.730454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.730467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.730502 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.730743 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.731297 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.732666 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.733038 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.733129 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.733140 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.733322 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.733361 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:06:33.733791 10334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00de0debd0343e18f1b38fb7c165c30" candidate_uuid: "8ce27fe404a24599af9812f6276a16b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469fc47ba0214a98a08e3d7631df2295" is_pre_election: true
I20260812 00:06:33.733918 10334 raft_consensus.cc:2454] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce27fe404a24599af9812f6276a16b3 in term 0.
I20260812 00:06:33.734079 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.730459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:58277 (local address 127.7.85.195:41305)
0812 00:06:33.730544 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.730547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.730605 (+ 58us) server_negotiation.cc:407] Connection header received
0812 00:06:33.730657 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.730658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.730667 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.730725 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:33.731551 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.732239 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.733218 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.733701 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.733810 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.733819 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.733972 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.734013 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:06:33.734088 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.730357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:41305 (local address 127.7.85.193:58277)
0812 00:06:33.730513 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.730525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.730606 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.731397 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.731398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.731400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:33.731462 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.731464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.732369 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.732370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.733078 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.733080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.733803 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.733804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.733883 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.733888 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.733951 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.733991 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260812 00:06:33.734113 9896 leader_election.cc:302] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469fc47ba0214a98a08e3d7631df2295, 8ce27fe404a24599af9812f6276a16b3; no voters:
I20260812 00:06:33.734232 10773 raft_consensus.cc:2781] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:33.734270 10773 raft_consensus.cc:482] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:33.734303 10773 raft_consensus.cc:3037] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.734362 10622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00de0debd0343e18f1b38fb7c165c30" candidate_uuid: "8ce27fe404a24599af9812f6276a16b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d271563be9ae4dbab4dd866fad59dabb" is_pre_election: true
I20260812 00:06:33.734503 10622 raft_consensus.cc:2454] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ce27fe404a24599af9812f6276a16b3 in term 0.
I20260812 00:06:33.735157 10773 raft_consensus.cc:504] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.735314 10773 leader_election.cc:288] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [CANDIDATE]: Term 1 election: Requested vote from peers 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267), d271563be9ae4dbab4dd866fad59dabb (127.7.85.195:41305)
I20260812 00:06:33.735607 10334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00de0debd0343e18f1b38fb7c165c30" candidate_uuid: "8ce27fe404a24599af9812f6276a16b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469fc47ba0214a98a08e3d7631df2295"
I20260812 00:06:33.735646 10622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a00de0debd0343e18f1b38fb7c165c30" candidate_uuid: "8ce27fe404a24599af9812f6276a16b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d271563be9ae4dbab4dd866fad59dabb"
I20260812 00:06:33.735899 10622 raft_consensus.cc:3037] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.736373 10334 raft_consensus.cc:3037] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:33.737005 10622 raft_consensus.cc:2454] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce27fe404a24599af9812f6276a16b3 in term 1.
I20260812 00:06:33.737419 9896 leader_election.cc:302] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ce27fe404a24599af9812f6276a16b3, d271563be9ae4dbab4dd866fad59dabb; no voters:
I20260812 00:06:33.737557 10773 raft_consensus.cc:2781] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:33.737736 10773 raft_consensus.cc:686] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [term 1 LEADER]: Becoming Leader. State: Replica: 8ce27fe404a24599af9812f6276a16b3, State: Running, Role: LEADER
I20260812 00:06:33.737831 10773 consensus_queue.cc:237] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } }
I20260812 00:06:33.737883 10334 raft_consensus.cc:2454] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ce27fe404a24599af9812f6276a16b3 in term 1.
I20260812 00:06:33.738515 10828 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: TxnStatusTablet state changed. Reason: New leader 8ce27fe404a24599af9812f6276a16b3. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.738628 10828 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:06:33.738950 10832 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 00:06:33.739202 10832 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 00:06:33.739305 9703 catalog_manager.cc:5495] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193). New cstate: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } health_report { overall_health: UNKNOWN } } }
I20260812 00:06:33.755129 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.752026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40293 (local address 127.0.0.1:35352)
0812 00:06:33.752212 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.752238 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.752363 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.752765 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.752767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.752769 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:33.752829 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.752842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.753595 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.753597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.754189 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.754190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.754559 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.754560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.754624 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.754643 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.754940 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.754951 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.755003 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.755056 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 00:06:33.755138 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.752232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.7.85.193:40293)
0812 00:06:33.752316 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.752319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.752330 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:33.752599 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.752602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.752633 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.752736 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:33.752923 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.753514 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.754293 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.754486 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.754562 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.754759 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.754871 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.754954 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.755006 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.755028 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:06:33.756842 10334 raft_consensus.cc:1261] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Refusing update from remote peer 8ce27fe404a24599af9812f6276a16b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:33.756856 10622 raft_consensus.cc:1261] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Refusing update from remote peer 8ce27fe404a24599af9812f6276a16b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:33.757205 10828 consensus_queue.cc:1035] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:33.757390 10773 consensus_queue.cc:1035] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:33.758067 10777 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: TxnStatusTablet state changed. Reason: New leader 8ce27fe404a24599af9812f6276a16b3. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.758059 10772 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: TxnStatusTablet state changed. Reason: New leader 8ce27fe404a24599af9812f6276a16b3. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.758172 10777 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.758180 10772 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.758801 10828 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.758890 10828 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:06:33.759027 10772 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.759104 10772 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 469fc47ba0214a98a08e3d7631df2295: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.759240 10828 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.759313 10828 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P 8ce27fe404a24599af9812f6276a16b3: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:06:33.759706 10777 tablet_replica.cc:439] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ce27fe404a24599af9812f6276a16b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 } } peers { permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 } } peers { permanent_uuid: "d271563be9ae4dbab4dd866fad59dabb" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 41305 } } }
I20260812 00:06:33.759785 10777 tablet_replica.cc:442] T a00de0debd0343e18f1b38fb7c165c30 P d271563be9ae4dbab4dd866fad59dabb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:33.767591 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.764158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40293 (local address 127.7.85.195:43565)
0812 00:06:33.764386 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.764401 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.764460 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.764590 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.764590 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.764592 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:33.764640 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.764642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.765697 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.765698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.766335 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.766337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.766921 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.766924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.767021 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.767056 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.767379 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.767384 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.767453 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.767517 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":120,"threads_started":1}
I20260812 00:06:33.767593 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.764293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:43565 (local address 127.7.85.193:40293)
0812 00:06:33.764363 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.764366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.764465 (+ 99us) server_negotiation.cc:407] Connection header received
0812 00:06:33.764501 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.764502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.764514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.764588 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:33.764679 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.765538 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.766427 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.766736 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.766822 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.767202 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.767241 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.767310 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.767469 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.767495 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:06:33.770457 10334 raft_consensus.cc:1261] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Refusing update from remote peer d271563be9ae4dbab4dd866fad59dabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:33.770468 10026 raft_consensus.cc:1261] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Refusing update from remote peer d271563be9ae4dbab4dd866fad59dabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:33.770802 10782 consensus_queue.cc:1035] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ce27fe404a24599af9812f6276a16b3" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:33.771148 10777 consensus_queue.cc:1035] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "469fc47ba0214a98a08e3d7631df2295" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:33.781662 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.778188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:44547 (local address 127.7.85.195:41305)
0812 00:06:33.778234 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.778237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.778365 (+ 128us) server_negotiation.cc:407] Connection header received
0812 00:06:33.778434 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.778435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.778449 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.778494 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:33.778655 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.779146 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.780835 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.781049 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.781128 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.781357 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.781396 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.781491 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.781553 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.781578 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:06:33.781736 10854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.778132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:41305 (local address 127.7.85.193:44547)
0812 00:06:33.778306 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:33.778318 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:33.778382 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:33.778504 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:33.778505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:33.778506 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:33.778576 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.778577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.779252 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.779253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.780740 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:33.780743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.781128 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.781129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:33.781238 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.781265 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:33.781498 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:33.781501 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:06:33.781548 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 00:06:33.781606 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260812 00:06:33.792148 9702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:33.792474 9702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:33.793812 9702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3519f272fb63493db2893d0ecd673731: Sending DeleteTablet for 3 replicas of tablet e28905461eb44a99927cd07e226c320e
I20260812 00:06:33.794288 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet e28905461eb44a99927cd07e226c320e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:33 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39868
I20260812 00:06:33.794262 10285 tablet_service.cc:1514] Processing DeleteTablet for tablet e28905461eb44a99927cd07e226c320e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:33 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:42234
I20260812 00:06:33.794432 9988 tablet_service.cc:1514] Processing DeleteTablet for tablet e28905461eb44a99927cd07e226c320e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:33 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:35346
I20260812 00:06:33.794469 10798 tablet_replica.cc:330] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: stopping tablet replica
I20260812 00:06:33.794586 10804 tablet_replica.cc:330] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: stopping tablet replica
I20260812 00:06:33.794628 10798 raft_consensus.cc:2229] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:33.794679 10804 raft_consensus.cc:2229] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:33.794735 10798 raft_consensus.cc:2258] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:33.794742 10804 raft_consensus.cc:2258] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:33.794847 10796 tablet_replica.cc:330] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: stopping tablet replica
I20260812 00:06:33.794934 10796 raft_consensus.cc:2229] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:33.794989 10796 raft_consensus.cc:2258] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:33.795573 10804 ts_tablet_manager.cc:1905] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:33.795828 10796 ts_tablet_manager.cc:1905] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:33.797894 10804 ts_tablet_manager.cc:1918] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:33.797914 10798 ts_tablet_manager.cc:1905] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:33.797988 10804 log.cc:1196] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/e28905461eb44a99927cd07e226c320e
I20260812 00:06:33.798256 10804 ts_tablet_manager.cc:1939] T e28905461eb44a99927cd07e226c320e P 8ce27fe404a24599af9812f6276a16b3: Deleting consensus metadata
I20260812 00:06:33.798657 9670 catalog_manager.cc:4841] TS 8ce27fe404a24599af9812f6276a16b3 (127.7.85.193:40293): tablet e28905461eb44a99927cd07e226c320e (table test-table [id=c13d68cf4f08466c8c9a652b30371dd5]) successfully deleted
I20260812 00:06:33.799733 10798 ts_tablet_manager.cc:1918] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:33.799809 10798 log.cc:1196] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/e28905461eb44a99927cd07e226c320e
I20260812 00:06:33.800040 10798 ts_tablet_manager.cc:1939] T e28905461eb44a99927cd07e226c320e P d271563be9ae4dbab4dd866fad59dabb: Deleting consensus metadata
I20260812 00:06:33.800312 10796 ts_tablet_manager.cc:1918] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:33.800390 10796 log.cc:1196] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/e28905461eb44a99927cd07e226c320e
I20260812 00:06:33.800542 9668 catalog_manager.cc:4841] TS d271563be9ae4dbab4dd866fad59dabb (127.7.85.195:41305): tablet e28905461eb44a99927cd07e226c320e (table test-table [id=c13d68cf4f08466c8c9a652b30371dd5]) successfully deleted
I20260812 00:06:33.800645 10796 ts_tablet_manager.cc:1939] T e28905461eb44a99927cd07e226c320e P 469fc47ba0214a98a08e3d7631df2295: Deleting consensus metadata
I20260812 00:06:33.801124 9668 catalog_manager.cc:4841] TS 469fc47ba0214a98a08e3d7631df2295 (127.7.85.194:34267): tablet e28905461eb44a99927cd07e226c320e (table test-table [id=c13d68cf4f08466c8c9a652b30371dd5]) successfully deleted
Aug 12 00:06:33 dist-test-slave-z8x9 krb5kdc[9622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493193, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:33.808743 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.795962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.7.85.193:40293)
0812 00:06:33.796031 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.796034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.797542 (+ 1508us) server_negotiation.cc:407] Connection header received
0812 00:06:33.797688 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.797689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.797705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.797740 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:33.799137 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.799964 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.801445 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.801639 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.801707 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.805431 (+ 3724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:33.805445 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:33.805446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:33.805455 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:33.807277 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.807801 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.807812 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.807813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.807887 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.808105 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.808106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.808106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.808270 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:33.808470 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.808521 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.808609 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 00:06:33.809092 10128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35362
I20260812 00:06:33.817737 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:33.810340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35760 (local address 127.7.85.254:46777)
0812 00:06:33.810393 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:33.810396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:33.810458 (+ 62us) server_negotiation.cc:407] Connection header received
0812 00:06:33.810538 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:33.810539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:33.810551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:33.810611 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:33.811252 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.811830 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.812601 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:33.812811 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:33.812897 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:33.814398 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:33.814399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:33.814400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:33.814408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:33.816033 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.816688 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.816689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.816690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.816768 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:33.817081 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:33.817083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:33.817083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:33.817311 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:33.817428 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:33.817536 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 00:06:33.817625 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 00:06:33.818001 9702 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35760
I20260812 00:06:33.818813 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9856
I20260812 00:06:33.853824 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10153
I20260812 00:06:33.907701 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10441
I20260812 00:06:33.940888 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9643
2026-08-12T00:06:33Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3162 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:34 dist-test-slave-z8x9 krb5kdc[10871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:34 dist-test-slave-z8x9 krb5kdc[10871](info): set up 2 sockets
Aug 12 00:06:34 dist-test-slave-z8x9 krb5kdc[10871](info): commencing operation
krb5kdc: starting...
W20260812 00:06:35.305506 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:35.462499 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33371
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:35.557292 10891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:35.557533 10891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:35.557556 10891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:35.559408 10891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:35.559479 10891 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 00:06:35.559501 10891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:35.559540 10891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:35.559562 10891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:35.559584 10891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:35.561780 10891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33371
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:35.562117 10891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:35.562402 10891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:35.568614 10906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:35.568742 10900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:35.568857 10891 server_base.cc:1034] running on GCE node
W20260812 00:06:35.568614 10901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:35.569288 10891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:35.569664 10891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:35.570888 10891 hybrid_clock.cc:648] HybridClock initialized: now 1786493195570900 us; error 58 us; skew 500 ppm
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:35.574476 10891 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:35.575067 10891 webserver.cc:460] Webserver started at http://127.7.85.254:36451/ using document root <none> and password file <none>
I20260812 00:06:35.575348 10891 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:35.575424 10891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:35.575582 10891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:35.576841 10891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "6c3045fc32974cbb82ba86792e7efc2d"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "039fa18eeab18293d44a4bd219ea7dcb"
server_key_iv: "7dcfbe8ffb4248a1a73bb8fc82e07f2c"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.577351 10891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "6c3045fc32974cbb82ba86792e7efc2d"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "039fa18eeab18293d44a4bd219ea7dcb"
server_key_iv: "7dcfbe8ffb4248a1a73bb8fc82e07f2c"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.578851 10891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:35.579787 10918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:35.580057 10891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:35.580179 10891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "6c3045fc32974cbb82ba86792e7efc2d"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "039fa18eeab18293d44a4bd219ea7dcb"
server_key_iv: "7dcfbe8ffb4248a1a73bb8fc82e07f2c"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.580305 10891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:35.588359 10891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:35.591374 10891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:35.591576 10891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:35.596884 10891 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33371
I20260812 00:06:35.596944 11037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33371 every 8 connection(s)
I20260812 00:06:35.597430 10891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:35.598202 11038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:35.599678 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10891
I20260812 00:06:35.599798 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:35.599968 7511 external_mini_cluster.cc:1428] Setting key 29b58ba4c09ba8b9fe6061f833c057e1
I20260812 00:06:35.601332 11038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Bootstrap starting.
I20260812 00:06:35.601967 11038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:35.602283 11038 log.cc:824] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:35.603124 11038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: No bootstrap required, opened a new log
I20260812 00:06:35.605522 11038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3045fc32974cbb82ba86792e7efc2d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33371 } }
I20260812 00:06:35.605767 11038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:35.605850 11038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3045fc32974cbb82ba86792e7efc2d, State: Initialized, Role: FOLLOWER
I20260812 00:06:35.606038 11038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3045fc32974cbb82ba86792e7efc2d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33371 } }
I20260812 00:06:35.606387 11038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:35.606654 11038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:35.606737 11038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 0 FOLLOWER]: Advancing to term 1
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:35.607511 11038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3045fc32974cbb82ba86792e7efc2d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33371 } }
I20260812 00:06:35.607651 11038 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3045fc32974cbb82ba86792e7efc2d; no voters:
I20260812 00:06:35.607864 11038 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:35.607918 11049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:35.608080 11049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [term 1 LEADER]: Becoming Leader. State: Replica: 6c3045fc32974cbb82ba86792e7efc2d, State: Running, Role: LEADER
I20260812 00:06:35.608168 11049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3045fc32974cbb82ba86792e7efc2d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33371 } }
I20260812 00:06:35.608170 11038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:35.609061 11054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3045fc32974cbb82ba86792e7efc2d. Latest consensus state: current_term: 1 leader_uuid: "6c3045fc32974cbb82ba86792e7efc2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3045fc32974cbb82ba86792e7efc2d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33371 } } }
I20260812 00:06:35.609153 11054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:35.609709 11063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:35.610330 11063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:35.610751 11051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c3045fc32974cbb82ba86792e7efc2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3045fc32974cbb82ba86792e7efc2d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33371 } } }
I20260812 00:06:35.611085 11051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:35.612241 11063 catalog_manager.cc:1347] Generated new cluster ID: 942f56ccb1c942808405b8e64937f539
I20260812 00:06:35.612252 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.600567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.7.85.254:33371)
0812 00:06:35.600786 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:35.600794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:35.600821 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:06:35.601410 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:35.601420 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:35.601492 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:35.601639 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:35.602353 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.603528 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.604285 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.604732 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.604868 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.607693 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:35.607697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:35.607698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:35.607714 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:35.610306 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:35.610777 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:35.610781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:35.610781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:35.610916 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:35.611182 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:35.611183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:35.611183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:35.611514 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:35.611693 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:35.611867 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 00:06:35.612036 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260812 00:06:35.612309 11063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:35.618502 11063 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:35.619125 11063 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:35.628265 11063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Generated new TSK 0
I20260812 00:06:35.628584 11063 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:35.669575 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33371
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:35.761147 11092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:35.761405 11092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:35.761432 11092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:35.762640 11092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:35.762686 11092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:35.762708 11092 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 00:06:35.762720 11092 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:06:35.762738 11092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:35.764257 11092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:33371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:35.764540 11092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:35.764794 11092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:35.770278 11100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:35.770301 11104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:35.770328 11101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:35.770535 11092 server_base.cc:1034] running on GCE node
I20260812 00:06:35.770766 11092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:35.770999 11092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:35.772359 11092 hybrid_clock.cc:648] HybridClock initialized: now 1786493195772256 us; error 113 us; skew 500 ppm
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:35.775282 11092 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:35.775811 11092 webserver.cc:460] Webserver started at http://127.7.85.193:34967/ using document root <none> and password file <none>
I20260812 00:06:35.776085 11092 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:35.776134 11092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:35.776237 11092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:35.777356 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "739d4dceabba9c7fd1b6d07a74efdd04"
server_key_iv: "88b5a8c2cc0d8a824716b3c5ec8230a4"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.777732 11092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "739d4dceabba9c7fd1b6d07a74efdd04"
server_key_iv: "88b5a8c2cc0d8a824716b3c5ec8230a4"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.779103 11092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:35.779949 11122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:35.780233 11092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:35.780359 11092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "739d4dceabba9c7fd1b6d07a74efdd04"
server_key_iv: "88b5a8c2cc0d8a824716b3c5ec8230a4"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.780438 11092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:35.797271 11092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:35.799752 11092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:35.799943 11092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:35.800558 11092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:35.800624 11092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:35.800678 11092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:35.800711 11092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:35.808941 11092 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:38561
I20260812 00:06:35.809034 11356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:38561 every 8 connection(s)
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:35.809480 11092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:35.813164 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.801267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:49867 (local address 127.7.85.254:33371)
0812 00:06:35.801339 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:35.801341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:35.801668 (+ 327us) server_negotiation.cc:407] Connection header received
0812 00:06:35.802398 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:35.802401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:35.802414 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:35.802474 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:35.803563 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.804437 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.806149 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.806457 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.806758 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.809988 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:35.809990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:35.809990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:35.810000 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:35.811596 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:35.812081 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:35.812082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:35.812083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:35.812149 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:35.812439 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:35.812440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:35.812440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:35.812577 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:35.812659 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:35.812968 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 00:06:35.813069 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:06:35.813402 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.801247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.7.85.193:49867)
0812 00:06:35.801512 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:35.801553 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:35.802143 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:35.802738 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:35.802743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:35.803135 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:35.803455 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:35.803461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.804561 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.804563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:35.806049 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:35.806052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.806774 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.806776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:35.806958 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.807635 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:35.807667 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:35.809868 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:35.811714 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:35.811723 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:35.811726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:35.811986 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:35.812218 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:35.812219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:35.812220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:35.812356 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:35.812647 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:35.812655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:35.812814 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 00:06:35.812931 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 00:06:35.813114 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":116,"threads_started":1}
I20260812 00:06:35.814069 11357 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33371
I20260812 00:06:35.814174 11357 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:35.814410 11357 heartbeater.cc:511] Master 127.7.85.254:33371 requested a full tablet report, sending...
I20260812 00:06:35.815524 10946 ts_manager.cc:194] Registered new tserver with Master: f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561)
I20260812 00:06:35.816358 10946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:49867
I20260812 00:06:35.816671 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11092
I20260812 00:06:35.816787 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:35.817013 7511 external_mini_cluster.cc:1428] Setting key 59b767e48190b655fb9cfa505ec5f72e
I20260812 00:06:35.823951 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.815379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.7.85.193:50727)
0812 00:06:35.815555 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:35.815567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:35.815649 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:35.815783 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:35.815784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:35.815984 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:35.816042 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:35.816044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.816941 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.816942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:35.817805 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:35.817808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.818220 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.818221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:35.818293 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.818850 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:35.818856 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:35.819196 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:35.821938 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:35.821940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:35.821940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:35.822151 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:35.822711 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:35.822713 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:35.822714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:35.822860 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:35.823549 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:35.823551 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:35.823619 (+ 68us) client_negotiation.cc:769] Sending connection context
0812 00:06:35.823722 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 00:06:35.823847 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
I20260812 00:06:35.824188 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.815456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:50727 (local address 127.7.85.254:33371)
0812 00:06:35.815520 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:35.815522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:35.815633 (+ 111us) server_negotiation.cc:407] Connection header received
0812 00:06:35.815718 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:35.815719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:35.815730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:35.815781 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:35.816122 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.816829 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.817905 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.818118 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.818218 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.819310 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:35.819312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:35.819312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:35.819320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:35.821803 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:35.822466 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:35.822467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:35.822468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:35.822548 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:35.823013 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:35.823017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:35.823017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:35.823394 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:35.823512 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:35.823809 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 00:06:35.824005 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:35.853828 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33371
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:35.947499 11381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:35.947739 11381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:35.947768 11381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:35.949102 11381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:35.949213 11381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:35.949239 11381 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 00:06:35.949254 11381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:06:35.949271 11381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:35.951308 11381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:33371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:35.951550 11381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:35.951762 11381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:35.957525 11392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:35.957533 11399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:35.957536 11393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:35.958225 11381 server_base.cc:1034] running on GCE node
I20260812 00:06:35.958487 11381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:35.958824 11381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:35.960273 11381 hybrid_clock.cc:648] HybridClock initialized: now 1786493195960248 us; error 48 us; skew 500 ppm
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:35.963272 11381 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:35.963788 11381 webserver.cc:460] Webserver started at http://127.7.85.194:33023/ using document root <none> and password file <none>
I20260812 00:06:35.964056 11381 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:35.964128 11381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:35.964278 11381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:35.965448 11381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "ce0ad754092145b5b76fd6f2b163b6aa"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "53344260070887e418fc6c96d2f86531"
server_key_iv: "04dfcc5ea2aca9bdab22c562d6336a24"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.965816 11381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "ce0ad754092145b5b76fd6f2b163b6aa"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "53344260070887e418fc6c96d2f86531"
server_key_iv: "04dfcc5ea2aca9bdab22c562d6336a24"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.967428 11381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:35.968492 11414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:35.968827 11381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:35.968978 11381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "ce0ad754092145b5b76fd6f2b163b6aa"
format_stamp: "Formatted at 2026-08-12 00:06:35 on dist-test-slave-z8x9"
server_key: "53344260070887e418fc6c96d2f86531"
server_key_iv: "04dfcc5ea2aca9bdab22c562d6336a24"
server_key_version: "encryptionkey@0"
I20260812 00:06:35.969065 11381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:35.986773 11381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:35.990070 11381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:35.990288 11381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:35.990983 11381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:35.991051 11381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:35.991088 11381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:35.991115 11381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:06:35 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:35.999846 11381 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:33069
I20260812 00:06:35.999892 11619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:33069 every 8 connection(s)
I20260812 00:06:36.000342 11381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:06:36.001029 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11381
I20260812 00:06:36.001230 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:36.001457 7511 external_mini_cluster.cc:1428] Setting key 791e684a2d22adce32d646bcf8d24f1b
I20260812 00:06:36.003768 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.991545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:46519 (local address 127.7.85.254:33371)
0812 00:06:35.991627 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:35.991630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:35.992012 (+ 382us) server_negotiation.cc:407] Connection header received
0812 00:06:35.992514 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:35.992517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:35.992542 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:35.992693 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:35.993590 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.994472 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.995581 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.995966 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.996062 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.999641 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:35.999643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:35.999643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:35.999654 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:36.001949 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.002503 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.002504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.002505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.002577 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.002924 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.002925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.002925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.003094 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:36.003169 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.003508 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.003608 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:06:36.003808 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:35.991505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.7.85.194:46519)
0812 00:06:35.991671 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:35.991714 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:35.992371 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:35.992804 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:35.992808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:35.993081 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:35.993458 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:35.993463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.994586 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.994587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:35.995501 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:35.995504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:35.996060 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:35.996061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:35.996241 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:35.997012 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:35.997024 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:35.999487 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:36.002035 (+ 2548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.002043 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.002047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.002413 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.002699 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.002700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.002700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.002841 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.003192 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:36.003199 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:36.003353 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.003458 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.003608 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260812 00:06:36.005010 11621 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33371
I20260812 00:06:36.005124 11621 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:36.005411 11621 heartbeater.cc:511] Master 127.7.85.254:33371 requested a full tablet report, sending...
I20260812 00:06:36.005919 10945 ts_manager.cc:194] Registered new tserver with Master: ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069)
I20260812 00:06:36.006492 10945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:46519
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
I20260812 00:06:36.013860 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.005636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.7.85.194:39953)
0812 00:06:36.005809 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.005820 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.005870 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.006053 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.006054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.006265 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:36.006333 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.006335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.007356 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.007356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.008004 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.008006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.008563 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.008564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.008646 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.009244 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:36.009255 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:36.009641 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:36.012150 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.012151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.012151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.012354 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.012737 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.012740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.012741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.012916 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.013490 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:36.013492 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:36.013550 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.013616 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.013749 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260812 00:06:36.013929 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.005741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:39953 (local address 127.7.85.254:33371)
0812 00:06:36.005844 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.005846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.005856 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:06:36.005920 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.005921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.005932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.005988 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:36.006421 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.007241 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.008095 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.008436 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.008518 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.009730 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:36.009732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:36.009732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:36.009740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:36.012031 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.012465 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.012466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.012467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.012546 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.013113 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.013114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.013114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.013405 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:36.013472 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.013727 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.013811 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:36.038211 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33371
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:36.131424 11633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:36.131659 11633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:36.131687 11633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:36.132788 11633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:36.132827 11633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:36.132848 11633 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 00:06:36.132860 11633 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:06:36.132879 11633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:36.134436 11633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:33371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:36.134651 11633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:36.134846 11633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:06:36.140559 11633 server_base.cc:1034] running on GCE node
W20260812 00:06:36.140513 11652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:36.140511 11648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:36.140524 11646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:36.141002 11633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:36.141319 11633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:36.142575 11633 hybrid_clock.cc:648] HybridClock initialized: now 1786493196142530 us; error 42 us; skew 500 ppm
Aug 12 00:06:36 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:36.146091 11633 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:36.146612 11633 webserver.cc:460] Webserver started at http://127.7.85.195:38031/ using document root <none> and password file <none>
I20260812 00:06:36.146865 11633 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:36.146940 11633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:36.147106 11633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:36.148240 11633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "fc6bb431854646afb9899cc8e367c78e"
format_stamp: "Formatted at 2026-08-12 00:06:36 on dist-test-slave-z8x9"
server_key: "5aa1a8a0de426dc14649adb96776b57e"
server_key_iv: "737babeb1f48286fcd0d345431f65a60"
server_key_version: "encryptionkey@0"
I20260812 00:06:36.148627 11633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "fc6bb431854646afb9899cc8e367c78e"
format_stamp: "Formatted at 2026-08-12 00:06:36 on dist-test-slave-z8x9"
server_key: "5aa1a8a0de426dc14649adb96776b57e"
server_key_iv: "737babeb1f48286fcd0d345431f65a60"
server_key_version: "encryptionkey@0"
I20260812 00:06:36.150254 11633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:36.151186 11660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:36.151463 11633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:36.151599 11633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "fc6bb431854646afb9899cc8e367c78e"
format_stamp: "Formatted at 2026-08-12 00:06:36 on dist-test-slave-z8x9"
server_key: "5aa1a8a0de426dc14649adb96776b57e"
server_key_iv: "737babeb1f48286fcd0d345431f65a60"
server_key_version: "encryptionkey@0"
I20260812 00:06:36.151702 11633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:36.176440 11633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:36.179102 11633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:36.179306 11633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:36.180011 11633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:36.180066 11633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:36.180102 11633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:36.180126 11633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:06:36 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493196, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:36.189188 11633 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:36225
I20260812 00:06:36.189333 11890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:36225 every 8 connection(s)
I20260812 00:06:36.189662 11633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:36.192570 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.180575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:57491 (local address 127.7.85.254:33371)
0812 00:06:36.180655 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.180659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.180956 (+ 297us) server_negotiation.cc:407] Connection header received
0812 00:06:36.181525 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.181526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.181538 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.181592 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:36.182253 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.182955 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.184486 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.184731 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.184809 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.188955 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:36.188957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:36.188958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:36.188968 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:36.191033 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.191500 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.191502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.191502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.191573 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.191851 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.191852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.191853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.191987 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:36.192068 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.192379 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.192473 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:06:36.192801 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.180611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.7.85.195:57491)
0812 00:06:36.180810 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.180853 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.181412 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.181674 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.181678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.181936 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:36.182144 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.182147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.183078 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.183080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.184393 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.184396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.184831 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.184832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.184983 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.185760 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:36.185780 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:36.188831 (+ 3051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:36.191139 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.191146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.191149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.191410 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.191649 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.191649 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.191650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.191771 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.192058 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:36.192066 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:36.192226 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.192340 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.192529 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260812 00:06:36.193446 11892 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33371
I20260812 00:06:36.193581 11892 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:36.193796 11892 heartbeater.cc:511] Master 127.7.85.254:33371 requested a full tablet report, sending...
I20260812 00:06:36.194283 10945 ts_manager.cc:194] Registered new tserver with Master: fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:36.194754 10945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:57491
I20260812 00:06:36.196208 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11633
I20260812 00:06:36.196305 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:36.196557 7511 external_mini_cluster.cc:1428] Setting key 708b828af46847eb6c6387934d5c9f54
I20260812 00:06:36.197403 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:06:36.202266 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.194593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.7.85.195:44027)
0812 00:06:36.194770 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.194782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.194839 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.195003 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.195003 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.195289 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:36.195415 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.195417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.196135 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.196137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.196853 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.196855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.197268 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.197269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.197362 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.197905 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:36.197910 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:36.198228 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:36.200186 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.200187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.200187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.200673 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.201279 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.201280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.201281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.201397 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.201878 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:36.201879 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:36.201968 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.202048 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.202167 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":124,"threads_started":1}
I20260812 00:06:36.202399 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.194646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:44027 (local address 127.7.85.254:33371)
0812 00:06:36.194879 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.194881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.194890 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:36.194911 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.194912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.194922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.194997 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:36.195528 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.196033 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.196939 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.197162 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.197269 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.198356 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:36.198357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:36.198358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:36.198366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:36.200086 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.200873 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.200877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.200878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.201059 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.201507 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.201509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.201509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.201731 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:36.201834 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.202115 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.202241 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
Aug 12 00:06:36 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:06:36 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:36.229753 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.220143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.7.85.254:33371)
0812 00:06:36.220343 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.220346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.220444 (+ 98us) server_negotiation.cc:407] Connection header received
0812 00:06:36.220552 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.220553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.220572 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.220639 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:36.221351 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.221863 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.222628 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.222816 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.222894 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.225727 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:36.225729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:36.225729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:36.225739 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:36.227852 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.228317 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.228318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.228319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.228386 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.228728 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.228730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.228730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.228872 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:36.228977 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.229335 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.229659 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260812 00:06:36.232069 10945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:36.233906 10945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:36.241554 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.237355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33069 (local address 127.0.0.1:54892)
0812 00:06:36.237817 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.237862 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.237953 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.238286 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.238290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.238295 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.238404 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.238407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.239342 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.239343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.240480 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.240482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.241218 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.241219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.241301 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.241317 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.241396 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.241454 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":24,"thread_start_us":76,"threads_started":1}
I20260812 00:06:36.241573 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.237585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.7.85.194:33069)
0812 00:06:36.237860 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.237864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.237993 (+ 129us) server_negotiation.cc:407] Connection header received
0812 00:06:36.238070 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.238081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.238096 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.238210 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.238489 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.239247 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.240544 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.241058 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.241189 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.241261 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.241426 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.241468 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":94,"threads_started":1}
I20260812 00:06:36.242311 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.237380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36225 (local address 127.0.0.1:55710)
0812 00:06:36.238088 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.238097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.238145 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.238389 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.238389 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.238391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.238439 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.238440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.239737 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.239738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.241026 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.241028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.242048 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.242049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.242112 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.242117 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.242178 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.242225 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":57,"thread_start_us":116,"threads_started":1}
I20260812 00:06:36.242319 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.237581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.7.85.193:38561)
0812 00:06:36.237768 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.237774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.237961 (+ 187us) server_negotiation.cc:407] Connection header received
0812 00:06:36.238085 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.238087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.238107 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.238187 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.238490 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.239646 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.241125 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.241865 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.241969 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.242029 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.242160 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.242215 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
I20260812 00:06:36.242336 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.237698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.7.85.195:36225)
0812 00:06:36.237942 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.237946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.238203 (+ 257us) server_negotiation.cc:407] Connection header received
0812 00:06:36.238245 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.238247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.238262 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.238331 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.238517 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.239634 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.241247 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.241955 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.242047 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.242106 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.242192 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.242226 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":87,"threads_started":1}
I20260812 00:06:36.242730 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.237355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38561 (local address 127.0.0.1:45280)
0812 00:06:36.237817 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.237862 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.237965 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.238286 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.238290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.238292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.238393 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.238396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.239738 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.239738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.241026 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.241028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.241976 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.241978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.242048 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.242054 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.242411 (+ 357us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.242460 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":107,"thread_start_us":102,"threads_started":1}
I20260812 00:06:36.243039 11497 tablet_service.cc:1467] Processing CreateTablet for tablet c4429752af0c4491956abe07598dd4e1 (DEFAULT_TABLE table=test-table [id=b413c115d5a14fb2bff5857ca34a7e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:36.243083 11752 tablet_service.cc:1467] Processing CreateTablet for tablet c4429752af0c4491956abe07598dd4e1 (DEFAULT_TABLE table=test-table [id=b413c115d5a14fb2bff5857ca34a7e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:36.243366 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4429752af0c4491956abe07598dd4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.243453 11752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4429752af0c4491956abe07598dd4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.243404 11218 tablet_service.cc:1467] Processing CreateTablet for tablet c4429752af0c4491956abe07598dd4e1 (DEFAULT_TABLE table=test-table [id=b413c115d5a14fb2bff5857ca34a7e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:36.244081 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4429752af0c4491956abe07598dd4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.246495 11937 tablet_bootstrap.cc:492] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Bootstrap starting.
I20260812 00:06:36.247417 11937 tablet_bootstrap.cc:654] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.247740 11937 log.cc:824] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:36.248108 11940 tablet_bootstrap.cc:492] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Bootstrap starting.
I20260812 00:06:36.248637 11938 tablet_bootstrap.cc:492] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Bootstrap starting.
I20260812 00:06:36.248827 11940 tablet_bootstrap.cc:654] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.248917 11937 tablet_bootstrap.cc:492] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: No bootstrap required, opened a new log
I20260812 00:06:36.248996 11937 ts_tablet_manager.cc:1397] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:06:36.249279 11940 log.cc:824] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:36.249635 11938 tablet_bootstrap.cc:654] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.250001 11938 log.cc:824] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:36.250054 11940 tablet_bootstrap.cc:492] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: No bootstrap required, opened a new log
I20260812 00:06:36.250113 11940 ts_tablet_manager.cc:1397] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:06:36.250815 11938 tablet_bootstrap.cc:492] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: No bootstrap required, opened a new log
I20260812 00:06:36.250892 11938 ts_tablet_manager.cc:1397] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 00:06:36.251005 11937 raft_consensus.cc:348] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.251168 11937 raft_consensus.cc:374] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.251202 11937 raft_consensus.cc:729] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.251329 11937 consensus_queue.cc:260] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.251616 11937 ts_tablet_manager.cc:1428] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:06:36.251658 11621 heartbeater.cc:503] Master 127.7.85.254:33371 was elected leader, sending a full tablet report...
I20260812 00:06:36.251945 11940 raft_consensus.cc:348] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.252107 11940 raft_consensus.cc:374] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.252146 11940 raft_consensus.cc:729] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f799fe4c8c93429ba722ef0d7f27fa4a, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.252275 11940 consensus_queue.cc:260] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.252542 11940 ts_tablet_manager.cc:1428] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 00:06:36.252700 11357 heartbeater.cc:503] Master 127.7.85.254:33371 was elected leader, sending a full tablet report...
I20260812 00:06:36.253103 11938 raft_consensus.cc:348] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.253302 11938 raft_consensus.cc:374] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.253342 11938 raft_consensus.cc:729] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.253445 11938 consensus_queue.cc:260] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.253669 11938 ts_tablet_manager.cc:1428] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:06:36.253973 11892 heartbeater.cc:503] Master 127.7.85.254:33371 was elected leader, sending a full tablet report...
I20260812 00:06:36.309844 11950 raft_consensus.cc:482] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:36.310194 11950 raft_consensus.cc:504] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
W20260812 00:06:36.310566 11359 tablet.cc:2369] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:36.310879 11950 leader_election.cc:288] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069)
I20260812 00:06:36.315416 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.311258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:33159 (local address 127.7.85.194:33069)
0812 00:06:36.311304 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.311306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.311484 (+ 178us) server_negotiation.cc:407] Connection header received
0812 00:06:36.311516 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.311517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.311528 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.311566 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.311741 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.312462 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.314335 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.314921 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.315029 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.315040 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.315286 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.315340 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:06:36.315418 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.311170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:56577 (local address 127.7.85.193:38561)
0812 00:06:36.311231 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.311233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.311247 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:06:36.311332 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.311334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.311346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.311408 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.311631 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.312464 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.314335 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.314934 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.315030 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.315040 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.315286 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.315336 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 00:06:36.315425 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.311110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38561 (local address 127.7.85.195:56577)
0812 00:06:36.311154 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.311201 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.311258 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.311439 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.311440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.311442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.311532 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.311534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.312694 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.312696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.314187 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.314190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.315048 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.315050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.315140 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.315148 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.315222 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.315268 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":67}
I20260812 00:06:36.315433 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.311112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33069 (local address 127.7.85.195:33159)
0812 00:06:36.311386 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.311396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.311448 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.311599 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.311600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.311602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.311668 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.311669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.312696 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.312696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.314187 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.314190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.315048 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.315050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.315141 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.315148 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.315222 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.315268 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":109,"thread_start_us":134,"threads_started":1}
I20260812 00:06:36.315909 11529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4429752af0c4491956abe07598dd4e1" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" is_pre_election: true
I20260812 00:06:36.315896 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4429752af0c4491956abe07598dd4e1" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" is_pre_election: true
I20260812 00:06:36.316092 11529 raft_consensus.cc:2454] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 0.
I20260812 00:06:36.316102 11258 raft_consensus.cc:2454] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 0.
I20260812 00:06:36.316370 11661 leader_election.cc:302] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f799fe4c8c93429ba722ef0d7f27fa4a, fc6bb431854646afb9899cc8e367c78e; no voters:
I20260812 00:06:36.316514 11950 raft_consensus.cc:2781] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:36.316606 11950 raft_consensus.cc:482] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:36.316645 11950 raft_consensus.cc:3037] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.317582 11950 raft_consensus.cc:504] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.317874 11950 leader_election.cc:288] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 election: Requested vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069)
I20260812 00:06:36.317927 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4429752af0c4491956abe07598dd4e1" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
I20260812 00:06:36.318037 11258 raft_consensus.cc:3037] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.318890 11258 raft_consensus.cc:2454] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 1.
I20260812 00:06:36.319267 11661 leader_election.cc:302] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f799fe4c8c93429ba722ef0d7f27fa4a, fc6bb431854646afb9899cc8e367c78e; no voters:
I20260812 00:06:36.319317 11529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4429752af0c4491956abe07598dd4e1" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0ad754092145b5b76fd6f2b163b6aa"
I20260812 00:06:36.319419 11950 raft_consensus.cc:2781] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:36.319417 11529 raft_consensus.cc:3037] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.319615 11950 raft_consensus.cc:686] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 1 LEADER]: Becoming Leader. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Running, Role: LEADER
I20260812 00:06:36.319723 11950 consensus_queue.cc:237] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.320453 11529 raft_consensus.cc:2454] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 1.
I20260812 00:06:36.320757 10945 catalog_manager.cc:5495] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e reported cstate change: term changed from 0 to 1, leader changed from <none> to fc6bb431854646afb9899cc8e367c78e (127.7.85.195). New cstate: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } health_report { overall_health: HEALTHY } } }
Aug 12 00:06:36 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:36.387123 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.378159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33371 (local address 127.0.0.1:59148)
0812 00:06:36.378503 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.378527 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.378658 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.379001 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.379002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.379573 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:36.379640 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.379643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.380478 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.380480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.381020 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.381022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.381432 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.381433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.381485 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.382134 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:36.382139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:36.383818 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:36.385776 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.385786 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.385789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.386033 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.386326 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:36.386327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:36.386327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:36.386420 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:36.386793 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:36.386801 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:36.386852 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.386920 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.387037 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":115,"threads_started":1}
I20260812 00:06:36.387244 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.378228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.7.85.254:33371)
0812 00:06:36.378307 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.378309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.378731 (+ 422us) server_negotiation.cc:407] Connection header received
0812 00:06:36.378776 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.378778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.378792 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.378894 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:36.379794 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.380328 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.381125 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.381330 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.381413 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.384022 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:36.384024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:36.384024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:36.384032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:36.385664 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.386116 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.386117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.386117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.386206 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:36.386524 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:36.386525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:36.386526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:36.386670 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:36.386738 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.386959 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.387096 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":48}
I20260812 00:06:36.390214 10945 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:59148:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 00:06:36.390560 10945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:36.393040 11218 tablet_service.cc:1467] Processing CreateTablet for tablet 2d780188db00426bb7a7f1765f28c078 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=85e1b4ed12d14c28b7e556915a982c2c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:06:36.393019 11497 tablet_service.cc:1467] Processing CreateTablet for tablet 2d780188db00426bb7a7f1765f28c078 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=85e1b4ed12d14c28b7e556915a982c2c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:06:36.393337 11752 tablet_service.cc:1467] Processing CreateTablet for tablet 2d780188db00426bb7a7f1765f28c078 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=85e1b4ed12d14c28b7e556915a982c2c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:06:36.393386 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d780188db00426bb7a7f1765f28c078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.393386 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d780188db00426bb7a7f1765f28c078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.393513 11752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d780188db00426bb7a7f1765f28c078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.394999 11938 tablet_bootstrap.cc:492] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: Bootstrap starting.
I20260812 00:06:36.395094 11940 tablet_bootstrap.cc:492] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: Bootstrap starting.
I20260812 00:06:36.395094 11937 tablet_bootstrap.cc:492] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: Bootstrap starting.
I20260812 00:06:36.395496 11938 tablet_bootstrap.cc:654] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.395540 11940 tablet_bootstrap.cc:654] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.395846 11937 tablet_bootstrap.cc:654] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.396330 11940 tablet_bootstrap.cc:492] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: No bootstrap required, opened a new log
I20260812 00:06:36.396414 11940 ts_tablet_manager.cc:1397] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:36.396559 11937 tablet_bootstrap.cc:492] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: No bootstrap required, opened a new log
I20260812 00:06:36.396595 11937 ts_tablet_manager.cc:1397] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:06:36.396615 11938 tablet_bootstrap.cc:492] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: No bootstrap required, opened a new log
I20260812 00:06:36.396605 11940 raft_consensus.cc:348] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.396654 11938 ts_tablet_manager.cc:1397] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 00:06:36.396689 11940 raft_consensus.cc:374] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.396716 11940 raft_consensus.cc:729] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f799fe4c8c93429ba722ef0d7f27fa4a, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.396718 11937 raft_consensus.cc:348] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.396776 11940 consensus_queue.cc:260] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.396796 11937 raft_consensus.cc:374] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.396821 11937 raft_consensus.cc:729] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.396812 11938 raft_consensus.cc:348] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.396888 11940 ts_tablet_manager.cc:1428] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:36.396898 11938 raft_consensus.cc:374] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.396870 11937 consensus_queue.cc:260] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.396921 11938 raft_consensus.cc:729] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.396970 11937 ts_tablet_manager.cc:1428] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:06:36.396966 11938 consensus_queue.cc:260] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.396946 11949 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.397045 11949 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:36.397068 11938 ts_tablet_manager.cc:1428] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:36.397121 11947 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.397267 11947 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:36.397471 11950 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.397567 11950 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 00:06:36.440377 11894 tablet.cc:2369] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:06:36.501335 11622 tablet.cc:2369] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:36.721243 11950 raft_consensus.cc:482] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:36.721423 11950 raft_consensus.cc:504] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.721690 11950 leader_election.cc:288] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069)
I20260812 00:06:36.722024 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d780188db00426bb7a7f1765f28c078" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" is_pre_election: true
I20260812 00:06:36.722038 11529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d780188db00426bb7a7f1765f28c078" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" is_pre_election: true
I20260812 00:06:36.722160 11258 raft_consensus.cc:2454] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 0.
I20260812 00:06:36.722240 11529 raft_consensus.cc:2454] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 0.
I20260812 00:06:36.722409 11661 leader_election.cc:302] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f799fe4c8c93429ba722ef0d7f27fa4a, fc6bb431854646afb9899cc8e367c78e; no voters:
I20260812 00:06:36.722530 11950 raft_consensus.cc:2781] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:36.722600 11950 raft_consensus.cc:482] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:36.722617 11950 raft_consensus.cc:3037] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.723371 11950 raft_consensus.cc:504] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.723528 11950 leader_election.cc:288] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 election: Requested vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069)
I20260812 00:06:36.723734 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d780188db00426bb7a7f1765f28c078" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
I20260812 00:06:36.723822 11258 raft_consensus.cc:3037] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.723799 11529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d780188db00426bb7a7f1765f28c078" candidate_uuid: "fc6bb431854646afb9899cc8e367c78e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce0ad754092145b5b76fd6f2b163b6aa"
I20260812 00:06:36.723937 11529 raft_consensus.cc:3037] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.724501 11258 raft_consensus.cc:2454] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 1.
I20260812 00:06:36.724691 11661 leader_election.cc:302] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f799fe4c8c93429ba722ef0d7f27fa4a, fc6bb431854646afb9899cc8e367c78e; no voters:
I20260812 00:06:36.724808 11950 raft_consensus.cc:2781] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:36.724882 11950 raft_consensus.cc:686] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [term 1 LEADER]: Becoming Leader. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Running, Role: LEADER
I20260812 00:06:36.724941 11950 consensus_queue.cc:237] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.724995 11529 raft_consensus.cc:2454] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6bb431854646afb9899cc8e367c78e in term 1.
I20260812 00:06:36.725241 11952 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: TxnStatusTablet state changed. Reason: New leader fc6bb431854646afb9899cc8e367c78e. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.725342 11952 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:06:36.725521 11969 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 00:06:36.725682 11969 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 00:06:36.725641 10946 catalog_manager.cc:5495] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e reported cstate change: term changed from 0 to 1, leader changed from <none> to fc6bb431854646afb9899cc8e367c78e (127.7.85.195). New cstate: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } health_report { overall_health: HEALTHY } } }
I20260812 00:06:36.759371 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.756283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.7.85.195:36225)
0812 00:06:36.756394 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.756396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.756406 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:06:36.756456 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.756457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.756475 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.756550 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:36.756715 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.757340 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.758478 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.758681 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.758772 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.759011 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.759130 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.759196 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.759275 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.759294 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260812 00:06:36.759409 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.756185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36225 (local address 127.0.0.1:55716)
0812 00:06:36.756273 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.756286 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.756331 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.756575 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.756576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.756578 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:36.756633 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.756635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.757419 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.757420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.758377 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.758380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.758799 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.758801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.758889 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.758909 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.759204 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.759213 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.759271 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.759323 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 00:06:36.761940 11529 raft_consensus.cc:1261] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Refusing update from remote peer fc6bb431854646afb9899cc8e367c78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:36.761951 11258 raft_consensus.cc:1261] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Refusing update from remote peer fc6bb431854646afb9899cc8e367c78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:36.762390 11950 consensus_queue.cc:1035] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:36.762529 11952 consensus_queue.cc:1035] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:36.763937 11947 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: TxnStatusTablet state changed. Reason: New leader fc6bb431854646afb9899cc8e367c78e. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.764037 11947 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:36.764344 11949 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: TxnStatusTablet state changed. Reason: New leader fc6bb431854646afb9899cc8e367c78e. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.764431 11949 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:36.765239 11950 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.765326 11950 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:06:36.765689 11977 mvcc.cc:205] Tried to move back new op lower bound from 7317476133933420544 to 7317476133785747456. Current Snapshot: MvccSnapshot[applied={T|T < 7317476133933420544}]
I20260812 00:06:36.766455 11949 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.766539 11949 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P f799fe4c8c93429ba722ef0d7f27fa4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:36.767338 11952 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.767438 11952 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P fc6bb431854646afb9899cc8e367c78e: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:06:36.768067 11947 tablet_replica.cc:439] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fc6bb431854646afb9899cc8e367c78e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } } }
I20260812 00:06:36.768178 11947 tablet_replica.cc:442] T 2d780188db00426bb7a7f1765f28c078 P ce0ad754092145b5b76fd6f2b163b6aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:06:36.772125 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.767692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.7.85.195:36225)
0812 00:06:36.767768 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.767770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.769370 (+ 1600us) server_negotiation.cc:407] Connection header received
0812 00:06:36.769417 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.769418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.769429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.769485 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:36.769615 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.770109 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.771264 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.771571 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.771637 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.771872 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.771907 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.771966 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.772046 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.772068 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:06:36.779022 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.775943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:51039 (local address 127.7.85.195:36225)
0812 00:06:36.775988 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.775990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.776319 (+ 329us) server_negotiation.cc:407] Connection header received
0812 00:06:36.776345 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.776345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.776354 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.776404 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:36.776537 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.777062 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.778252 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.778468 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.778523 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.778759 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.778791 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.778836 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.778917 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.778934 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:06:36.779110 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.775855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36225 (local address 127.7.85.195:51039)
0812 00:06:36.776212 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.776224 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.776294 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.776408 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.776409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.776410 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:36.776480 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.776482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.777150 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.777152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.778141 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.778144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.778545 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.778546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.778633 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.778651 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.778857 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.778859 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.778915 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.778958 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":99,"threads_started":1}
I20260812 00:06:36.782119 11258 raft_consensus.cc:1261] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Refusing update from remote peer fc6bb431854646afb9899cc8e367c78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:36.782119 11529 raft_consensus.cc:1261] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Refusing update from remote peer fc6bb431854646afb9899cc8e367c78e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:36.782389 12007 consensus_queue.cc:1035] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:36.782544 11952 consensus_queue.cc:1035] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:36.783797 11975 mvcc.cc:205] Tried to move back new op lower bound from 7317476134017855488 to 7317476132126355456. Current Snapshot: MvccSnapshot[applied={T|T < 7317476134017855488}]
I20260812 00:06:36.784466 11976 mvcc.cc:205] Tried to move back new op lower bound from 7317476134017855488 to 7317476132126355456. Current Snapshot: MvccSnapshot[applied={T|T < 7317476134017855488}]
I20260812 00:06:36.797430 10946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:36.797544 10946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:36.798359 10946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Sending DeleteTablet for 3 replicas of tablet c4429752af0c4491956abe07598dd4e1
I20260812 00:06:36.798867 11218 tablet_service.cc:1514] Processing DeleteTablet for tablet c4429752af0c4491956abe07598dd4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:36 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45280
I20260812 00:06:36.798929 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet c4429752af0c4491956abe07598dd4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:36 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:54892
I20260812 00:06:36.799351 12021 tablet_replica.cc:330] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: stopping tablet replica
I20260812 00:06:36.799603 12021 raft_consensus.cc:2229] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:36.799831 12021 raft_consensus.cc:2258] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:36.800133 11752 tablet_service.cc:1514] Processing DeleteTablet for tablet c4429752af0c4491956abe07598dd4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:36 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:55710
I20260812 00:06:36.800387 12021 ts_tablet_manager.cc:1905] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:36.800436 12022 tablet_replica.cc:330] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: stopping tablet replica
I20260812 00:06:36.800437 10946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:36.800621 10946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:36.800997 12022 raft_consensus.cc:2229] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:36.801256 12022 raft_consensus.cc:2258] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:36.801745 12022 ts_tablet_manager.cc:1905] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:36.802083 11218 tablet_service.cc:1467] Processing CreateTablet for tablet b8c182a1fa6c4609af19b992b80e4ab3 (DEFAULT_TABLE table=test-table [id=482642f41b1b4883b975741b7c32c8d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:36.802217 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c182a1fa6c4609af19b992b80e4ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.802639 12021 ts_tablet_manager.cc:1918] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:36.802724 12021 log.cc:1196] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/c4429752af0c4491956abe07598dd4e1
I20260812 00:06:36.803027 12021 ts_tablet_manager.cc:1939] T c4429752af0c4491956abe07598dd4e1 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting consensus metadata
I20260812 00:06:36.803836 12022 ts_tablet_manager.cc:1918] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:36.803953 12022 log.cc:1196] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/c4429752af0c4491956abe07598dd4e1
I20260812 00:06:36.804103 11940 tablet_bootstrap.cc:492] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Bootstrap starting.
I20260812 00:06:36.804487 12022 ts_tablet_manager.cc:1939] T c4429752af0c4491956abe07598dd4e1 P fc6bb431854646afb9899cc8e367c78e: Deleting consensus metadata
I20260812 00:06:36.804643 11940 tablet_bootstrap.cc:654] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.805217 10922 catalog_manager.cc:4841] TS fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225): tablet c4429752af0c4491956abe07598dd4e1 (table test-table [id=b413c115d5a14fb2bff5857ca34a7e04]) successfully deleted
I20260812 00:06:36.805459 10926 catalog_manager.cc:4841] TS ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069): tablet c4429752af0c4491956abe07598dd4e1 (table test-table [id=b413c115d5a14fb2bff5857ca34a7e04]) successfully deleted
I20260812 00:06:36.805583 11940 tablet_bootstrap.cc:492] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: No bootstrap required, opened a new log
I20260812 00:06:36.805603 11497 tablet_service.cc:1467] Processing CreateTablet for tablet b8c182a1fa6c4609af19b992b80e4ab3 (DEFAULT_TABLE table=test-table [id=482642f41b1b4883b975741b7c32c8d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:36.805644 11940 ts_tablet_manager.cc:1397] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:06:36.805713 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c182a1fa6c4609af19b992b80e4ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.805812 11940 raft_consensus.cc:348] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.805909 11940 raft_consensus.cc:374] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.805931 11940 raft_consensus.cc:729] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f799fe4c8c93429ba722ef0d7f27fa4a, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.805972 11940 consensus_queue.cc:260] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.806063 11940 ts_tablet_manager.cc:1428] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 00:06:36.807042 11937 tablet_bootstrap.cc:492] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Bootstrap starting.
I20260812 00:06:36.807296 11752 tablet_service.cc:1467] Processing CreateTablet for tablet b8c182a1fa6c4609af19b992b80e4ab3 (DEFAULT_TABLE table=test-table [id=482642f41b1b4883b975741b7c32c8d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:36.807426 11752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8c182a1fa6c4609af19b992b80e4ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:36.807494 11937 tablet_bootstrap.cc:654] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.808236 11937 tablet_bootstrap.cc:492] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: No bootstrap required, opened a new log
I20260812 00:06:36.808298 11937 ts_tablet_manager.cc:1397] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:36.808462 11937 raft_consensus.cc:348] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.808545 11937 raft_consensus.cc:374] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.808569 11937 raft_consensus.cc:729] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.808624 11937 consensus_queue.cc:260] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.808722 11937 ts_tablet_manager.cc:1428] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:06:36.808873 11938 tablet_bootstrap.cc:492] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Bootstrap starting.
I20260812 00:06:36.809289 12016 tablet_replica.cc:330] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: stopping tablet replica
I20260812 00:06:36.809414 12016 raft_consensus.cc:2229] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:36.809478 12016 raft_consensus.cc:2258] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:36.809846 12016 ts_tablet_manager.cc:1905] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:36.811861 12016 ts_tablet_manager.cc:1918] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:36.811942 12016 log.cc:1196] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/c4429752af0c4491956abe07598dd4e1
I20260812 00:06:36.812223 12016 ts_tablet_manager.cc:1939] T c4429752af0c4491956abe07598dd4e1 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting consensus metadata
I20260812 00:06:36.812669 10919 catalog_manager.cc:4841] TS f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561): tablet c4429752af0c4491956abe07598dd4e1 (table test-table [id=b413c115d5a14fb2bff5857ca34a7e04]) successfully deleted
I20260812 00:06:36.812901 11938 tablet_bootstrap.cc:654] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:36.813498 11938 tablet_bootstrap.cc:492] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: No bootstrap required, opened a new log
I20260812 00:06:36.813570 11938 ts_tablet_manager.cc:1397] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.000s
I20260812 00:06:36.813745 11938 raft_consensus.cc:348] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.813927 11938 raft_consensus.cc:374] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:36.813982 11938 raft_consensus.cc:729] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Initialized, Role: FOLLOWER
I20260812 00:06:36.814052 11938 consensus_queue.cc:260] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.814190 11938 ts_tablet_manager.cc:1428] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 00:06:36.849292 11947 raft_consensus.cc:482] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:36.849432 11947 raft_consensus.cc:504] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.849819 11947 leader_election.cc:288] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:36.853853 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.850233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:59241 (local address 127.7.85.195:36225)
0812 00:06:36.850282 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.850284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.850296 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:06:36.850365 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.850366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.850376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.850447 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.850603 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.851398 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.852766 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.853269 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.853352 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.853362 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.853707 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.853763 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:06:36.853992 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.850013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36225 (local address 127.7.85.194:59241)
0812 00:06:36.850221 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.850231 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.850291 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.850469 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.850470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.850473 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.850539 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.850541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.851538 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.851539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.852635 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.852639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.853362 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.853363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.853432 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.853436 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.853688 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.853869 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":47,"thread_start_us":112,"threads_started":1}
I20260812 00:06:36.854332 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c182a1fa6c4609af19b992b80e4ab3" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6bb431854646afb9899cc8e367c78e" is_pre_election: true
I20260812 00:06:36.854425 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.850013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38561 (local address 127.7.85.194:36057)
0812 00:06:36.850212 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.850226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.850291 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.850543 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.850544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.850547 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:36.850627 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.850639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.851523 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.851525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.852975 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.852979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.853844 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.853846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.854218 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.854225 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.854296 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.854345 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":228,"thread_start_us":76,"threads_started":1}
I20260812 00:06:36.854486 11795 raft_consensus.cc:2454] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 0.
I20260812 00:06:36.854516 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.850095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:36057 (local address 127.7.85.193:38561)
0812 00:06:36.850391 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.850393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.850408 (+ 15us) server_negotiation.cc:407] Connection header received
0812 00:06:36.850453 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.850454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.850466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.850531 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:36.850725 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.851420 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.853084 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.853724 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.853797 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.853806 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.854383 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.854438 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":59,"threads_started":1}
I20260812 00:06:36.854678 11418 leader_election.cc:302] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce0ad754092145b5b76fd6f2b163b6aa, fc6bb431854646afb9899cc8e367c78e; no voters:
I20260812 00:06:36.854794 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c182a1fa6c4609af19b992b80e4ab3" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" is_pre_election: true
I20260812 00:06:36.854880 11258 raft_consensus.cc:2454] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 0.
I20260812 00:06:36.855103 11947 raft_consensus.cc:2781] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:36.855170 11947 raft_consensus.cc:482] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:36.855193 11947 raft_consensus.cc:3037] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.856235 11947 raft_consensus.cc:504] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.856407 11947 leader_election.cc:288] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 election: Requested vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:36.856662 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c182a1fa6c4609af19b992b80e4ab3" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6bb431854646afb9899cc8e367c78e"
I20260812 00:06:36.856761 11795 raft_consensus.cc:3037] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.856886 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8c182a1fa6c4609af19b992b80e4ab3" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
I20260812 00:06:36.856972 11258 raft_consensus.cc:3037] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:36.857705 11258 raft_consensus.cc:2454] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 1.
I20260812 00:06:36.857913 11416 leader_election.cc:302] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce0ad754092145b5b76fd6f2b163b6aa, f799fe4c8c93429ba722ef0d7f27fa4a; no voters:
I20260812 00:06:36.858063 11947 raft_consensus.cc:2781] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:36.858217 11795 raft_consensus.cc:2454] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 1.
I20260812 00:06:36.858285 11947 raft_consensus.cc:686] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 LEADER]: Becoming Leader. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Running, Role: LEADER
I20260812 00:06:36.858364 11947 consensus_queue.cc:237] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:36.859215 10943 catalog_manager.cc:5495] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194). New cstate: current_term: 1 leader_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } health_report { overall_health: UNKNOWN } } }
I20260812 00:06:36.876492 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.873390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54896 (local address 127.7.85.194:33069)
0812 00:06:36.873599 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.873601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.873612 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:36.873658 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.873659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.873679 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.873752 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:36.873915 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.874517 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.875545 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.875779 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.875843 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.876027 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.876169 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.876228 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.876308 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.876338 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":148,"threads_started":1}
I20260812 00:06:36.883502 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.880280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53709 (local address 127.7.85.195:36225)
0812 00:06:36.880372 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.880375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.880627 (+ 252us) server_negotiation.cc:407] Connection header received
0812 00:06:36.880723 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.880724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.880739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.880815 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:36.881031 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.881784 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.882627 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.882957 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.883030 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.883204 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.883240 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.883293 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.883387 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.883415 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:06:36.883991 12047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.880089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36225 (local address 127.7.85.194:53709)
0812 00:06:36.880444 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.880462 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.880645 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.880875 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.880876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.880879 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:36.880951 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.880953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.881898 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.881899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.882534 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.882540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.883049 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.883050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.883099 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.883118 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.883327 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.883329 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.883376 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.883416 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":260,"threads_started":1}
I20260812 00:06:36.886677 11258 raft_consensus.cc:1261] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Refusing update from remote peer ce0ad754092145b5b76fd6f2b163b6aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:36.886721 11795 raft_consensus.cc:1261] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Refusing update from remote peer ce0ad754092145b5b76fd6f2b163b6aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:36.887014 12037 consensus_queue.cc:1035] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:36.887156 11947 consensus_queue.cc:1035] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:36.895597 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.893002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:51505 (local address 127.7.85.194:33069)
0812 00:06:36.893043 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:36.893045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:36.893055 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:06:36.893084 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:36.893084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:36.893098 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:36.893205 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:36.893324 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.893822 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.894878 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.895203 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.895259 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.895409 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.895436 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.895468 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:36.895521 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 00:06:36.895538 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:06:36.895645 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:36.892941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33069 (local address 127.7.85.195:51505)
0812 00:06:36.892982 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:36.892994 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:36.893033 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:36.893209 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:36.893210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:36.893211 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:06:36.893260 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.893262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.893891 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.893892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.894797 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:36.894800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:36.895275 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:36.895276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:36.895336 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:36.895343 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:06:36.895473 (+ 130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:06:36.895474 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 00:06:36.895519 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 00:06:36.895563 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 00:06:36.896112 11976 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 00:06:38.425905 10943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:38.426040 10943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:38.427506 10943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Sending DeleteTablet for 3 replicas of tablet b8c182a1fa6c4609af19b992b80e4ab3
I20260812 00:06:38.427873 11749 tablet_service.cc:1514] Processing DeleteTablet for tablet b8c182a1fa6c4609af19b992b80e4ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:38 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:55710
I20260812 00:06:38.427996 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet b8c182a1fa6c4609af19b992b80e4ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:38 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:54892
I20260812 00:06:38.428025 11218 tablet_service.cc:1514] Processing DeleteTablet for tablet b8c182a1fa6c4609af19b992b80e4ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:38 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45280
I20260812 00:06:38.428282 12104 tablet_replica.cc:330] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: stopping tablet replica
I20260812 00:06:38.428354 12104 raft_consensus.cc:2229] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:38.428493 12104 raft_consensus.cc:2258] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:38.428629 12101 tablet_replica.cc:330] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: stopping tablet replica
I20260812 00:06:38.428689 12101 raft_consensus.cc:2229] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:38.428732 12101 raft_consensus.cc:2258] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:38.428994 12104 ts_tablet_manager.cc:1905] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:38.429024 12101 ts_tablet_manager.cc:1905] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:38.431571 12097 tablet_replica.cc:330] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: stopping tablet replica
I20260812 00:06:38.431646 12097 raft_consensus.cc:2229] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:38.431681 12097 raft_consensus.cc:2258] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:38.431845 12104 ts_tablet_manager.cc:1918] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:06:38.431875 12101 ts_tablet_manager.cc:1918] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:06:38.431936 12101 log.cc:1196] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/b8c182a1fa6c4609af19b992b80e4ab3
I20260812 00:06:38.431988 12097 ts_tablet_manager.cc:1905] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:38.432123 12104 log.cc:1196] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/b8c182a1fa6c4609af19b992b80e4ab3
I20260812 00:06:38.432299 12101 ts_tablet_manager.cc:1939] T b8c182a1fa6c4609af19b992b80e4ab3 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting consensus metadata
I20260812 00:06:38.432688 12104 ts_tablet_manager.cc:1939] T b8c182a1fa6c4609af19b992b80e4ab3 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting consensus metadata
I20260812 00:06:38.433068 10919 catalog_manager.cc:4841] TS f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561): tablet b8c182a1fa6c4609af19b992b80e4ab3 (table test-table [id=482642f41b1b4883b975741b7c32c8d0]) successfully deleted
I20260812 00:06:38.433218 10926 catalog_manager.cc:4841] TS ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069): tablet b8c182a1fa6c4609af19b992b80e4ab3 (table test-table [id=482642f41b1b4883b975741b7c32c8d0]) successfully deleted
I20260812 00:06:38.434899 12097 ts_tablet_manager.cc:1918] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:06:38.434964 12097 log.cc:1196] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/b8c182a1fa6c4609af19b992b80e4ab3
I20260812 00:06:38.435621 12097 ts_tablet_manager.cc:1939] T b8c182a1fa6c4609af19b992b80e4ab3 P fc6bb431854646afb9899cc8e367c78e: Deleting consensus metadata
I20260812 00:06:38.437445 10922 catalog_manager.cc:4841] TS fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225): tablet b8c182a1fa6c4609af19b992b80e4ab3 (table test-table [id=482642f41b1b4883b975741b7c32c8d0]) successfully deleted
Aug 12 00:06:38 dist-test-slave-z8x9 krb5kdc[10871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493198, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:06:38 dist-test-slave-z8x9 krb5kdc[10871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493198, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:38.470444 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:38.454270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.7.85.254:33371)
0812 00:06:38.455057 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:38.455060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:38.455170 (+ 110us) server_negotiation.cc:407] Connection header received
0812 00:06:38.455379 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:38.455380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:38.455401 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:38.455650 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:38.456604 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:38.457159 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:38.458051 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:38.458338 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:38.458514 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:38.466952 (+ 8438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:38.466956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:38.466957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:38.466973 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:38.468890 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:38.469523 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:38.469525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:38.469525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:38.469598 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:38.469846 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:38.469847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:38.469847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:38.469996 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:38.470085 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:38.470256 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 00:06:38.470364 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":668,"threads_started":1}
I20260812 00:06:38.472357 10943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:38.472573 10943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:38.474298 11218 tablet_service.cc:1467] Processing CreateTablet for tablet 4c62a8d945394a81a980269884fb97b5 (DEFAULT_TABLE table=test-table [id=618afe9b39b84256baa0d2a3c777ba63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:38.474344 11749 tablet_service.cc:1467] Processing CreateTablet for tablet 4c62a8d945394a81a980269884fb97b5 (DEFAULT_TABLE table=test-table [id=618afe9b39b84256baa0d2a3c777ba63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:38.474341 11497 tablet_service.cc:1467] Processing CreateTablet for tablet 4c62a8d945394a81a980269884fb97b5 (DEFAULT_TABLE table=test-table [id=618afe9b39b84256baa0d2a3c777ba63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:38.474483 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c62a8d945394a81a980269884fb97b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:38.474483 11749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c62a8d945394a81a980269884fb97b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:38.474509 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c62a8d945394a81a980269884fb97b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:38.475785 12128 tablet_bootstrap.cc:492] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Bootstrap starting.
I20260812 00:06:38.475966 12133 tablet_bootstrap.cc:492] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Bootstrap starting.
I20260812 00:06:38.476229 12135 tablet_bootstrap.cc:492] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Bootstrap starting.
I20260812 00:06:38.476320 12128 tablet_bootstrap.cc:654] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:38.476342 12133 tablet_bootstrap.cc:654] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:38.476568 12135 tablet_bootstrap.cc:654] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:38.477114 12128 tablet_bootstrap.cc:492] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: No bootstrap required, opened a new log
I20260812 00:06:38.477255 12128 ts_tablet_manager.cc:1397] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:38.477267 12133 tablet_bootstrap.cc:492] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: No bootstrap required, opened a new log
I20260812 00:06:38.477321 12133 ts_tablet_manager.cc:1397] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:38.477325 12135 tablet_bootstrap.cc:492] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: No bootstrap required, opened a new log
I20260812 00:06:38.477366 12135 ts_tablet_manager.cc:1397] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:38.477434 12128 raft_consensus.cc:348] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.477484 12135 raft_consensus.cc:348] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.477483 12133 raft_consensus.cc:348] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.477550 12133 raft_consensus.cc:374] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:38.477550 12135 raft_consensus.cc:374] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:38.477573 12133 raft_consensus.cc:729] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f799fe4c8c93429ba722ef0d7f27fa4a, State: Initialized, Role: FOLLOWER
I20260812 00:06:38.477573 12135 raft_consensus.cc:729] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Initialized, Role: FOLLOWER
I20260812 00:06:38.477626 12135 consensus_queue.cc:260] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.477664 12133 consensus_queue.cc:260] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.477627 12128 raft_consensus.cc:374] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:38.477746 12128 raft_consensus.cc:729] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Initialized, Role: FOLLOWER
I20260812 00:06:38.477734 12135 ts_tablet_manager.cc:1428] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:38.477876 12133 ts_tablet_manager.cc:1428] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:38.477870 12128 consensus_queue.cc:260] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.478516 12128 ts_tablet_manager.cc:1428] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:38.640837 12054 raft_consensus.cc:482] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:38.641027 12054 raft_consensus.cc:504] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.641340 12054 leader_election.cc:288] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:38.641616 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c62a8d945394a81a980269884fb97b5" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" is_pre_election: true
I20260812 00:06:38.641634 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c62a8d945394a81a980269884fb97b5" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6bb431854646afb9899cc8e367c78e" is_pre_election: true
I20260812 00:06:38.641737 11258 raft_consensus.cc:2454] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 0.
I20260812 00:06:38.641780 11795 raft_consensus.cc:2454] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 0.
I20260812 00:06:38.642011 11416 leader_election.cc:302] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce0ad754092145b5b76fd6f2b163b6aa, f799fe4c8c93429ba722ef0d7f27fa4a; no voters:
I20260812 00:06:38.642143 12054 raft_consensus.cc:2781] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:38.642216 12054 raft_consensus.cc:482] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:38.642233 12054 raft_consensus.cc:3037] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:38.643311 12054 raft_consensus.cc:504] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.643501 12054 leader_election.cc:288] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 election: Requested vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:38.643676 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c62a8d945394a81a980269884fb97b5" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
I20260812 00:06:38.643760 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c62a8d945394a81a980269884fb97b5" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6bb431854646afb9899cc8e367c78e"
I20260812 00:06:38.643800 11258 raft_consensus.cc:3037] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:38.643841 11795 raft_consensus.cc:3037] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:38.644667 11258 raft_consensus.cc:2454] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 1.
I20260812 00:06:38.644747 11795 raft_consensus.cc:2454] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 1.
I20260812 00:06:38.644948 11416 leader_election.cc:302] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce0ad754092145b5b76fd6f2b163b6aa, f799fe4c8c93429ba722ef0d7f27fa4a; no voters:
I20260812 00:06:38.645079 12054 raft_consensus.cc:2781] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:38.645157 12054 raft_consensus.cc:686] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 LEADER]: Becoming Leader. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Running, Role: LEADER
I20260812 00:06:38.645259 12054 consensus_queue.cc:237] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.646296 10943 catalog_manager.cc:5495] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194). New cstate: current_term: 1 leader_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } health_report { overall_health: UNKNOWN } } }
I20260812 00:06:38.670775 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:38.666497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54898 (local address 127.7.85.194:33069)
0812 00:06:38.666691 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:38.666693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:38.666702 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:38.666773 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:38.666774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:38.666786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:38.666857 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:38.667020 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:38.667855 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:38.669718 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:38.670097 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:38.670201 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:38.670395 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:38.670461 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:38.670521 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:38.670595 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 00:06:38.670623 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
I20260812 00:06:38.674129 11795 raft_consensus.cc:1261] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Refusing update from remote peer ce0ad754092145b5b76fd6f2b163b6aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:38.674185 11258 raft_consensus.cc:1261] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Refusing update from remote peer ce0ad754092145b5b76fd6f2b163b6aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:38.674423 12037 consensus_queue.cc:1035] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:38.674652 12037 consensus_queue.cc:1035] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:38.686481 10943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:38.686610 10943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:38.687567 10943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Sending DeleteTablet for 3 replicas of tablet 4c62a8d945394a81a980269884fb97b5
I20260812 00:06:38.687927 11749 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c62a8d945394a81a980269884fb97b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:38 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:55710
I20260812 00:06:38.688005 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c62a8d945394a81a980269884fb97b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:38 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:54892
I20260812 00:06:38.688059 11218 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c62a8d945394a81a980269884fb97b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:38 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45280
I20260812 00:06:38.688117 12104 tablet_replica.cc:330] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: stopping tablet replica
I20260812 00:06:38.688171 12104 raft_consensus.cc:2229] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:38.688177 12097 tablet_replica.cc:330] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: stopping tablet replica
I20260812 00:06:38.688239 12104 raft_consensus.cc:2258] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:38.688246 12097 raft_consensus.cc:2229] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:38.688176 10943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 00:06:38.688288 12097 raft_consensus.cc:2258] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 00:06:38.688328 10943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:38.688372 12101 tablet_replica.cc:330] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: stopping tablet replica
I20260812 00:06:38.688431 12101 raft_consensus.cc:2229] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:38.688480 12101 raft_consensus.cc:2258] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:38.688858 12104 ts_tablet_manager.cc:1905] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:38.688928 12101 ts_tablet_manager.cc:1905] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:38.689726 11497 tablet_service.cc:1467] Processing CreateTablet for tablet aab4520556c6452f8317416b3bcbacb0 (DEFAULT_TABLE table=test-table [id=fac6c7b7236340fdb4874d4b6990b001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:38.689894 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab4520556c6452f8317416b3bcbacb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:38.690555 11218 tablet_service.cc:1467] Processing CreateTablet for tablet aab4520556c6452f8317416b3bcbacb0 (DEFAULT_TABLE table=test-table [id=fac6c7b7236340fdb4874d4b6990b001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:38.690714 11749 tablet_service.cc:1467] Processing CreateTablet for tablet aab4520556c6452f8317416b3bcbacb0 (DEFAULT_TABLE table=test-table [id=fac6c7b7236340fdb4874d4b6990b001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:38.690841 11749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab4520556c6452f8317416b3bcbacb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:38.690851 12104 ts_tablet_manager.cc:1918] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:38.690908 12104 log.cc:1196] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/4c62a8d945394a81a980269884fb97b5
I20260812 00:06:38.691200 12104 ts_tablet_manager.cc:1939] T 4c62a8d945394a81a980269884fb97b5 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting consensus metadata
I20260812 00:06:38.691341 12128 tablet_bootstrap.cc:492] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Bootstrap starting.
I20260812 00:06:38.691812 12128 tablet_bootstrap.cc:654] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:38.692302 10926 catalog_manager.cc:4841] TS ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069): tablet 4c62a8d945394a81a980269884fb97b5 (table test-table [id=618afe9b39b84256baa0d2a3c777ba63]) successfully deleted
I20260812 00:06:38.692420 12135 tablet_bootstrap.cc:492] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Bootstrap starting.
I20260812 00:06:38.692574 12128 tablet_bootstrap.cc:492] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: No bootstrap required, opened a new log
I20260812 00:06:38.692643 12128 ts_tablet_manager.cc:1397] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:38.692653 12101 ts_tablet_manager.cc:1918] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:38.692710 12101 log.cc:1196] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/4c62a8d945394a81a980269884fb97b5
I20260812 00:06:38.692780 12135 tablet_bootstrap.cc:654] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:38.692780 12128 raft_consensus.cc:348] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.692881 12128 raft_consensus.cc:374] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:38.692909 12128 raft_consensus.cc:729] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Initialized, Role: FOLLOWER
I20260812 00:06:38.692960 12101 ts_tablet_manager.cc:1939] T 4c62a8d945394a81a980269884fb97b5 P f799fe4c8c93429ba722ef0d7f27fa4a: Deleting consensus metadata
I20260812 00:06:38.692958 12128 consensus_queue.cc:260] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.693101 12128 ts_tablet_manager.cc:1428] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 00:06:38.693181 11218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab4520556c6452f8317416b3bcbacb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:38.694595 12133 tablet_bootstrap.cc:492] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a: Bootstrap starting.
I20260812 00:06:38.694902 10919 catalog_manager.cc:4841] TS f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561): tablet 4c62a8d945394a81a980269884fb97b5 (table test-table [id=618afe9b39b84256baa0d2a3c777ba63]) successfully deleted
I20260812 00:06:38.695020 12097 ts_tablet_manager.cc:1905] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:38.695312 12133 tablet_bootstrap.cc:654] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:38.696003 12133 tablet_bootstrap.cc:492] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a: No bootstrap required, opened a new log
I20260812 00:06:38.696082 12133 ts_tablet_manager.cc:1397] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:06:38.696295 12133 raft_consensus.cc:348] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.696382 12133 raft_consensus.cc:374] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:38.696408 12133 raft_consensus.cc:729] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f799fe4c8c93429ba722ef0d7f27fa4a, State: Initialized, Role: FOLLOWER
I20260812 00:06:38.696462 12133 consensus_queue.cc:260] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.696570 12133 ts_tablet_manager.cc:1428] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:38.696755 12097 ts_tablet_manager.cc:1918] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:06:38.696825 12097 log.cc:1196] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/4c62a8d945394a81a980269884fb97b5
I20260812 00:06:38.697042 12097 ts_tablet_manager.cc:1939] T 4c62a8d945394a81a980269884fb97b5 P fc6bb431854646afb9899cc8e367c78e: Deleting consensus metadata
I20260812 00:06:38.697381 12135 tablet_bootstrap.cc:492] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: No bootstrap required, opened a new log
I20260812 00:06:38.697436 12135 ts_tablet_manager.cc:1397] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260812 00:06:38.697544 12135 raft_consensus.cc:348] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.697624 12135 raft_consensus.cc:374] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:38.697590 10922 catalog_manager.cc:4841] TS fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225): tablet 4c62a8d945394a81a980269884fb97b5 (table test-table [id=618afe9b39b84256baa0d2a3c777ba63]) successfully deleted
I20260812 00:06:38.697645 12135 raft_consensus.cc:729] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6bb431854646afb9899cc8e367c78e, State: Initialized, Role: FOLLOWER
I20260812 00:06:38.697721 12135 consensus_queue.cc:260] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.697830 12135 ts_tablet_manager.cc:1428] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 00:06:38.760571 12037 raft_consensus.cc:482] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:38.760715 12037 raft_consensus.cc:504] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.761240 12037 leader_election.cc:288] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:38.761520 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab4520556c6452f8317416b3bcbacb0" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" is_pre_election: true
I20260812 00:06:38.761550 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab4520556c6452f8317416b3bcbacb0" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6bb431854646afb9899cc8e367c78e" is_pre_election: true
I20260812 00:06:38.761644 11795 raft_consensus.cc:2454] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 0.
I20260812 00:06:38.761644 11258 raft_consensus.cc:2454] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 0.
I20260812 00:06:38.761838 11418 leader_election.cc:302] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce0ad754092145b5b76fd6f2b163b6aa, fc6bb431854646afb9899cc8e367c78e; no voters:
I20260812 00:06:38.761940 12037 raft_consensus.cc:2781] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:38.761986 12037 raft_consensus.cc:482] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:38.762012 12037 raft_consensus.cc:3037] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:38.762765 12037 raft_consensus.cc:504] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.762908 12037 leader_election.cc:288] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 election: Requested vote from peers f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561), fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225)
I20260812 00:06:38.763085 11795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab4520556c6452f8317416b3bcbacb0" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6bb431854646afb9899cc8e367c78e"
I20260812 00:06:38.763092 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab4520556c6452f8317416b3bcbacb0" candidate_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a"
I20260812 00:06:38.763185 11795 raft_consensus.cc:3037] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:38.763185 11258 raft_consensus.cc:3037] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:38.763852 11258 raft_consensus.cc:2454] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 1.
I20260812 00:06:38.763857 11795 raft_consensus.cc:2454] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce0ad754092145b5b76fd6f2b163b6aa in term 1.
I20260812 00:06:38.764053 11416 leader_election.cc:302] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce0ad754092145b5b76fd6f2b163b6aa, f799fe4c8c93429ba722ef0d7f27fa4a; no voters:
I20260812 00:06:38.764169 12037 raft_consensus.cc:2781] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:38.764370 12037 raft_consensus.cc:686] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 LEADER]: Becoming Leader. State: Replica: ce0ad754092145b5b76fd6f2b163b6aa, State: Running, Role: LEADER
I20260812 00:06:38.764464 12037 consensus_queue.cc:237] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } }
I20260812 00:06:38.765257 10943 catalog_manager.cc:5495] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa reported cstate change: term changed from 0 to 1, leader changed from <none> to ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194). New cstate: current_term: 1 leader_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce0ad754092145b5b76fd6f2b163b6aa" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 } health_report { overall_health: UNKNOWN } } }
I20260812 00:06:38.772564 11795 raft_consensus.cc:1261] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Refusing update from remote peer ce0ad754092145b5b76fd6f2b163b6aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:38.772614 11258 raft_consensus.cc:1261] T aab4520556c6452f8317416b3bcbacb0 P f799fe4c8c93429ba722ef0d7f27fa4a [term 1 FOLLOWER]: Refusing update from remote peer ce0ad754092145b5b76fd6f2b163b6aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:38.772897 12037 consensus_queue.cc:1035] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f799fe4c8c93429ba722ef0d7f27fa4a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:38.773021 12161 consensus_queue.cc:1035] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6bb431854646afb9899cc8e367c78e" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:06:38.778009 12067 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 00:06:40.303761 10946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:40.303897 10946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:40.305356 10946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c3045fc32974cbb82ba86792e7efc2d: Sending DeleteTablet for 3 replicas of tablet aab4520556c6452f8317416b3bcbacb0
I20260812 00:06:40.305753 11218 tablet_service.cc:1514] Processing DeleteTablet for tablet aab4520556c6452f8317416b3bcbacb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:40 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45280
I20260812 00:06:40.305744 11749 tablet_service.cc:1514] Processing DeleteTablet for tablet aab4520556c6452f8317416b3bcbacb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:40 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:55710
I20260812 00:06:40.305907 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11092
I20260812 00:06:40.309681 12209 tablet_replica.cc:330] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: stopping tablet replica
I20260812 00:06:40.309723 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet aab4520556c6452f8317416b3bcbacb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:40 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:54892
I20260812 00:06:40.309804 12209 raft_consensus.cc:2229] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:40.309875 12209 raft_consensus.cc:2258] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:40.309996 12213 tablet_replica.cc:330] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: stopping tablet replica
I20260812 00:06:40.310105 12213 raft_consensus.cc:2229] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:40.310210 12213 raft_consensus.cc:2258] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:40.313881 12209 ts_tablet_manager.cc:1905] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:40.316534 12209 ts_tablet_manager.cc:1918] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:06:40.316643 12209 log.cc:1196] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/aab4520556c6452f8317416b3bcbacb0
I20260812 00:06:40.316910 12209 ts_tablet_manager.cc:1939] T aab4520556c6452f8317416b3bcbacb0 P fc6bb431854646afb9899cc8e367c78e: Deleting consensus metadata
I20260812 00:06:40.318867 12213 ts_tablet_manager.cc:1905] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:40.320614 12213 ts_tablet_manager.cc:1918] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:06:40.320717 12213 log.cc:1196] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/aab4520556c6452f8317416b3bcbacb0
I20260812 00:06:40.320978 12213 ts_tablet_manager.cc:1939] T aab4520556c6452f8317416b3bcbacb0 P ce0ad754092145b5b76fd6f2b163b6aa: Deleting consensus metadata
I20260812 00:06:40.321698 10926 catalog_manager.cc:4841] TS ce0ad754092145b5b76fd6f2b163b6aa (127.7.85.194:33069): tablet aab4520556c6452f8317416b3bcbacb0 (table test-table [id=fac6c7b7236340fdb4874d4b6990b001]) successfully deleted
I20260812 00:06:40.324337 10922 catalog_manager.cc:4841] TS fc6bb431854646afb9899cc8e367c78e (127.7.85.195:36225): tablet aab4520556c6452f8317416b3bcbacb0 (table test-table [id=fac6c7b7236340fdb4874d4b6990b001]) successfully deleted
W20260812 00:06:40.330145 10919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:38561 (error 108)
I20260812 00:06:40.330628 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11381
I20260812 00:06:40.343713 12216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:40.331067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38561 (local address 127.0.0.1:45296)
0812 00:06:40.343145 (+ 12078us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:40.343635 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12003,"thread_start_us":99,"threads_started":1}
W20260812 00:06:40.343991 10919 catalog_manager.cc:4584] TS f799fe4c8c93429ba722ef0d7f27fa4a (127.7.85.193:38561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aab4520556c6452f8317416b3bcbacb0: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38561: connect: Connection refused (error 111)
I20260812 00:06:40.354259 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11633
I20260812 00:06:40.377527 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10891
2026-08-12T00:06:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6444 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:40 dist-test-slave-z8x9 krb5kdc[12222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:40 dist-test-slave-z8x9 krb5kdc[12222](info): set up 2 sockets
Aug 12 00:06:40 dist-test-slave-z8x9 krb5kdc[12222](info): commencing operation
krb5kdc: starting...
W20260812 00:06:41.788118 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.365s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:41 dist-test-slave-z8x9 krb5kdc[12222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:41.929160 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:42875
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:42875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:42.017264 12239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:42.017490 12239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:42.017514 12239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:42.018648 12239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:42.018684 12239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:42.018699 12239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:42.018712 12239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:42.018724 12239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:42.020197 12239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:42875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:42875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:42.020402 12239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:42.020581 12239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:42.026312 12246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.026330 12248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.026335 12245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:42.026405 12239 server_base.cc:1034] running on GCE node
I20260812 00:06:42.026904 12239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:42.027283 12239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:42.028475 12239 hybrid_clock.cc:648] HybridClock initialized: now 1786493202028462 us; error 49 us; skew 500 ppm
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:42.031611 12239 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:42.032232 12239 webserver.cc:460] Webserver started at http://127.7.85.254:44047/ using document root <none> and password file <none>
I20260812 00:06:42.032490 12239 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:42.032567 12239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:42.032740 12239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:42.033969 12239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "d9713dc603054f8f97ef46b4ec46f0e4"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "ddbc775c1140a37949f56f19842faea1"
server_key_iv: "79553236a563457d69a0675d2f00bd71"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.034359 12239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "d9713dc603054f8f97ef46b4ec46f0e4"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "ddbc775c1140a37949f56f19842faea1"
server_key_iv: "79553236a563457d69a0675d2f00bd71"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.036252 12239 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 00:06:42.037309 12264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.037545 12239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:42.037663 12239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "d9713dc603054f8f97ef46b4ec46f0e4"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "ddbc775c1140a37949f56f19842faea1"
server_key_iv: "79553236a563457d69a0675d2f00bd71"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.037756 12239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:42.061007 12239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:42.063825 12239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:42.064060 12239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:42.068616 12239 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:42875
I20260812 00:06:42.068651 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:42875 every 8 connection(s)
I20260812 00:06:42.069051 12239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:42.070030 12375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:42.072523 12375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4: Bootstrap starting.
I20260812 00:06:42.073160 12375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:42.073493 12375 log.cc:824] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:42.074302 12375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4: No bootstrap required, opened a new log
I20260812 00:06:42.075693 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12239
I20260812 00:06:42.075804 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:42.076005 7511 external_mini_cluster.cc:1428] Setting key f7965d763b6a895363df4533ae05848b
I20260812 00:06:42.075971 12375 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42875 } }
I20260812 00:06:42.076170 12375 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:42.076228 12375 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9713dc603054f8f97ef46b4ec46f0e4, State: Initialized, Role: FOLLOWER
I20260812 00:06:42.076387 12375 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42875 } }
I20260812 00:06:42.076495 12375 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:42.076531 12375 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:42.076612 12375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:42.077654 12375 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42875 } }
I20260812 00:06:42.077777 12375 leader_election.cc:302] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9713dc603054f8f97ef46b4ec46f0e4; no voters:
I20260812 00:06:42.077970 12375 leader_election.cc:288] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:42.078035 12386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:42.078250 12386 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [term 1 LEADER]: Becoming Leader. State: Replica: d9713dc603054f8f97ef46b4ec46f0e4, State: Running, Role: LEADER
I20260812 00:06:42.078297 12375 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:42.078433 12386 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42875 } }
I20260812 00:06:42.078855 12393 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9713dc603054f8f97ef46b4ec46f0e4. Latest consensus state: current_term: 1 leader_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42875 } } }
I20260812 00:06:42.079018 12393 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:42.079645 12407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:42.079914 12394 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9713dc603054f8f97ef46b4ec46f0e4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42875 } } }
I20260812 00:06:42.079995 12394 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:42.080255 12407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:42.082710 12407 catalog_manager.cc:1347] Generated new cluster ID: 701cad5f37494806971899c152319308
I20260812 00:06:42.082779 12407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:42.089316 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.076574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.7.85.254:42875)
0812 00:06:42.076785 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:42.076796 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:42.076835 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:06:42.077457 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:42.077468 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:42.077573 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:42.077748 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:42.078756 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.080325 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.081311 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.081726 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.081847 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.085000 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:42.085005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:42.085006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:42.085020 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:42.087335 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.087920 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.087924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.087924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.088057 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.088291 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.088291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.088292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.088604 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:42.088749 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:42.088908 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 00:06:42.089049 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260812 00:06:42.104274 12407 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:42.104861 12407 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:42.112792 12407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9713dc603054f8f97ef46b4ec46f0e4: Generated new TSK 0
I20260812 00:06:42.113072 12407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:42.152439 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:42875
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:42.250777 12437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:42.251020 12437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:42.251046 12437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:42.252519 12437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:42.252575 12437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:42.252605 12437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:42.255546 12437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:42875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:42.255872 12437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:42.256168 12437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:42.261799 12451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.261935 12447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:42.261997 12437 server_base.cc:1034] running on GCE node
W20260812 00:06:42.261799 12449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:42.262411 12437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:42.262696 12437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:42.263911 12437 hybrid_clock.cc:648] HybridClock initialized: now 1786493202263838 us; error 84 us; skew 500 ppm
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:42.267004 12437 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:42.267556 12437 webserver.cc:460] Webserver started at http://127.7.85.193:33277/ using document root <none> and password file <none>
I20260812 00:06:42.267810 12437 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:42.267884 12437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:42.268046 12437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:42.269841 12437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "c5ef3b8dd3a34d72846abf8d27f136ad"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "08fe49eff74c554103b6cffa1dad6615"
server_key_iv: "d58e397470e17e86eab1a3a8c7ae0c54"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.270337 12437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "c5ef3b8dd3a34d72846abf8d27f136ad"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "08fe49eff74c554103b6cffa1dad6615"
server_key_iv: "d58e397470e17e86eab1a3a8c7ae0c54"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.272058 12437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:42.273022 12469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.273437 12437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:42.273692 12437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "c5ef3b8dd3a34d72846abf8d27f136ad"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "08fe49eff74c554103b6cffa1dad6615"
server_key_iv: "d58e397470e17e86eab1a3a8c7ae0c54"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.273885 12437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:42.294350 12437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:42.297402 12437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:42.297618 12437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:42.297901 12437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:42.298269 12437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:42.298328 12437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.298398 12437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:42.298427 12437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.305598 12437 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:38417
I20260812 00:06:42.305721 12714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:38417 every 8 connection(s)
I20260812 00:06:42.306223 12437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:42.309161 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12437
I20260812 00:06:42.309356 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:42.309545 7511 external_mini_cluster.cc:1428] Setting key 22d463c5dd667f6b299ce5d037874c3f
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:42.317870 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.307233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:44715 (local address 127.7.85.254:42875)
0812 00:06:42.307317 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:42.307319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:42.307575 (+ 256us) server_negotiation.cc:407] Connection header received
0812 00:06:42.308045 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:42.308047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:42.308061 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:42.308130 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:42.309496 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.310190 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.311262 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.311480 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.311555 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.314437 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:42.314439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:42.314440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:42.314448 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:42.316246 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.316732 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.316733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.316733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.316805 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.317073 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.317074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.317074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.317291 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:42.317369 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:42.317672 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 00:06:42.317768 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 00:06:42.318092 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.307197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:42875 (local address 127.7.85.193:44715)
0812 00:06:42.307394 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:42.307444 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:42.307945 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:42.308310 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:42.308314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:42.309001 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:42.309333 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:42.309339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.310302 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.310303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:42.311141 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:42.311143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.311581 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.311583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:42.311728 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.312564 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:42.312576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:42.314326 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:42.316379 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:42.316387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:42.316390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:42.316643 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:42.316885 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:42.316886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:42.316886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:42.317003 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:42.317371 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:42.317379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:42.317531 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 00:06:42.317648 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 00:06:42.317833 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260812 00:06:42.318784 12716 heartbeater.cc:347] Connected to a master server at 127.7.85.254:42875
I20260812 00:06:42.319000 12716 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:42.319681 12716 heartbeater.cc:511] Master 127.7.85.254:42875 requested a full tablet report, sending...
I20260812 00:06:42.320628 12300 ts_manager.cc:194] Registered new tserver with Master: c5ef3b8dd3a34d72846abf8d27f136ad (127.7.85.193:38417)
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:06:42.321581 12300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:44715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:42.351967 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:42875
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:42.442744 12722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:42.442981 12722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:42.443007 12722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:42.444126 12722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:42.444173 12722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:42.444200 12722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:42.445660 12722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:42875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:42.445870 12722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:42.446067 12722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:42.451478 12736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.451488 12738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.451496 12733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:42.451933 12722 server_base.cc:1034] running on GCE node
I20260812 00:06:42.452172 12722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:42.452455 12722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:42.453804 12722 hybrid_clock.cc:648] HybridClock initialized: now 1786493202453770 us; error 52 us; skew 500 ppm
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:42.456527 12722 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:42.457033 12722 webserver.cc:460] Webserver started at http://127.7.85.194:43221/ using document root <none> and password file <none>
I20260812 00:06:42.457360 12722 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:42.457436 12722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:42.457598 12722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:42.458804 12722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "7b418418b221427a979f54fcf4515eca"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "8d55edf171baba201705e2ff4ee8bc77"
server_key_iv: "feab597d0de7a031b0b955bb234966ff"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.459189 12722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "7b418418b221427a979f54fcf4515eca"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "8d55edf171baba201705e2ff4ee8bc77"
server_key_iv: "feab597d0de7a031b0b955bb234966ff"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.461334 12722 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 00:06:42.462240 12748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.462463 12722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:42.462584 12722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "7b418418b221427a979f54fcf4515eca"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "8d55edf171baba201705e2ff4ee8bc77"
server_key_iv: "feab597d0de7a031b0b955bb234966ff"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.462669 12722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:42.477329 12722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:42.480350 12722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:42.480535 12722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:42.480810 12722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:42.481160 12722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:42.481254 12722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.481309 12722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:42.481338 12722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.488171 12722 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:32849
I20260812 00:06:42.488214 12982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:32849 every 8 connection(s)
I20260812 00:06:42.488610 12722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:42.498667 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12722
I20260812 00:06:42.498813 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:42.499023 7511 external_mini_cluster.cc:1428] Setting key a77fc7db5b90900a3d2fc8d564c2965d
I20260812 00:06:42.499151 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.489470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53479 (local address 127.7.85.254:42875)
0812 00:06:42.489540 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:42.489542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:42.489733 (+ 191us) server_negotiation.cc:407] Connection header received
0812 00:06:42.490241 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:42.490243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:42.490257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:42.490335 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:42.491382 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.491920 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.493088 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.493337 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.493400 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.495793 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:42.495795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:42.495795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:42.495803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:42.497365 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.497851 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.497852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.497852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.497923 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.498316 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.498318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.498318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.498452 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:42.498623 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:42.498956 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 00:06:42.499055 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 00:06:42.499380 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.489409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:42875 (local address 127.7.85.194:53479)
0812 00:06:42.489589 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:42.489635 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:42.490118 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:42.490419 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:42.490422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:42.491019 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:42.491219 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:42.491223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.492030 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.492031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:42.492964 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:42.492968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.493420 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.493421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:42.493536 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.494086 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:42.494095 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:42.495627 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:42.497468 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:42.497476 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:42.497478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:42.497742 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:42.498061 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:42.498062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:42.498062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:42.498222 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:42.498615 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:42.498623 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:42.498776 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 00:06:42.498913 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 00:06:42.499093 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260812 00:06:42.500041 12983 heartbeater.cc:347] Connected to a master server at 127.7.85.254:42875
I20260812 00:06:42.500164 12983 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:42.500372 12983 heartbeater.cc:511] Master 127.7.85.254:42875 requested a full tablet report, sending...
I20260812 00:06:42.500856 12300 ts_manager.cc:194] Registered new tserver with Master: 7b418418b221427a979f54fcf4515eca (127.7.85.194:32849)
I20260812 00:06:42.501372 12300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:53479
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:42.532124 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:42875
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:44459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:42.624639 12999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:42.624873 12999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:42.624900 12999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:42.626142 12999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:42.626233 12999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:42.626264 12999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:42.627707 12999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:42875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:42.627909 12999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:42.628168 12999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:42.633466 13014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.633466 13012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:42.633502 13009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:42.634042 12999 server_base.cc:1034] running on GCE node
I20260812 00:06:42.634295 12999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:42.634555 12999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:42.635720 12999 hybrid_clock.cc:648] HybridClock initialized: now 1786493202635700 us; error 31 us; skew 500 ppm
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:42.638626 12999 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:42.639158 12999 webserver.cc:460] Webserver started at http://127.7.85.195:36585/ using document root <none> and password file <none>
I20260812 00:06:42.639400 12999 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:42.639473 12999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:42.639633 12999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:42.640936 12999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "04b852e8e122462ba010b55dcd36fe2f"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "863c87bf13d3af2d7c25cf130e350cee"
server_key_iv: "f063b0217d1dca25e163086f2ecf2fa7"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.641340 12999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "04b852e8e122462ba010b55dcd36fe2f"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "863c87bf13d3af2d7c25cf130e350cee"
server_key_iv: "f063b0217d1dca25e163086f2ecf2fa7"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.642822 12999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:42.643654 13026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.643838 12999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:42.643952 12999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "04b852e8e122462ba010b55dcd36fe2f"
format_stamp: "Formatted at 2026-08-12 00:06:42 on dist-test-slave-z8x9"
server_key: "863c87bf13d3af2d7c25cf130e350cee"
server_key_iv: "f063b0217d1dca25e163086f2ecf2fa7"
server_key_version: "encryptionkey@0"
I20260812 00:06:42.644038 12999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:42.660344 12999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:42.663601 12999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:42.663781 12999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:42.664095 12999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:42.664463 12999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:42.664516 12999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.664552 12999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:42.664587 12999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:42.671270 12999 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:35367
I20260812 00:06:42.671362 13263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:35367 every 8 connection(s)
I20260812 00:06:42.671677 12999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:42.678877 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12999
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[12222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:42.679026 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:42.679256 7511 external_mini_cluster.cc:1428] Setting key ac16ad9539f98507560fe539241f26c4
I20260812 00:06:42.683050 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.672919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:58593 (local address 127.7.85.254:42875)
0812 00:06:42.672980 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:42.672983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:42.673345 (+ 362us) server_negotiation.cc:407] Connection header received
0812 00:06:42.673920 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:42.673921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:42.673935 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:42.673985 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:42.674968 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.675478 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.676493 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.676687 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.676765 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.679635 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:42.679637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:42.679637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:42.679646 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:42.681428 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.681898 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.681899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.681900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.681972 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:42.682295 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:42.682296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:42.682296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:42.682428 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:42.682499 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:42.682894 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 00:06:42.682980 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:06:42.683244 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:42.672898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:42875 (local address 127.7.85.195:58593)
0812 00:06:42.673078 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:42.673117 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:42.673800 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:42.674060 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:42.674064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:42.674657 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:42.674864 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:42.674869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.675580 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.675582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:42.676387 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:42.676390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:42.676775 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:42.676776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:42.676911 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:42.677588 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:42.677597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:42.679490 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:42.681520 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:42.681527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:42.681530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:42.681807 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:42.682070 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:42.682071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:42.682071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:42.682191 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:42.682601 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:42.682608 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:42.682735 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 00:06:42.682840 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 00:06:42.682996 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260812 00:06:42.683966 13265 heartbeater.cc:347] Connected to a master server at 127.7.85.254:42875
I20260812 00:06:42.684099 13265 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:42.684335 13265 heartbeater.cc:511] Master 127.7.85.254:42875 requested a full tablet report, sending...
I20260812 00:06:42.684810 12300 ts_manager.cc:194] Registered new tserver with Master: 04b852e8e122462ba010b55dcd36fe2f (127.7.85.195:35367)
I20260812 00:06:42.685160 12300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:58593
I20260812 00:06:42.691473 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 00:06:42.704814 12385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:42.703469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.7.85.254:42875)
0812 00:06:42.703545 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:42.703548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:42.703772 (+ 224us) server_negotiation.cc:407] Connection header received
0812 00:06:42.703811 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:42.703812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:42.703847 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:42.703918 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:42.704731 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46672: BlockingRecv error: recv got EOF from 127.0.0.1:46672 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:06:42.705850 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12437
I20260812 00:06:42.729439 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12722
I20260812 00:06:42.755388 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12999
I20260812 00:06:42.774426 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12239
2026-08-12T00:06:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2381 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[13293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[13293](info): set up 2 sockets
Aug 12 00:06:42 dist-test-slave-z8x9 krb5kdc[13293](info): commencing operation
krb5kdc: starting...
W20260812 00:06:44.142992 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:44 dist-test-slave-z8x9 krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:44.284759 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40239
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:38063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:40239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:44.373911 13320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:44.374136 13320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:44.374161 13320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:44.375279 13320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:44.375319 13320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:44.375334 13320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:44.375347 13320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:44.375360 13320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:44.376874 13320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:38063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:40239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:44.377097 13320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:44.377383 13320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:44.382721 13340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:44.382731 13333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:44.382730 13332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:44.383064 13320 server_base.cc:1034] running on GCE node
I20260812 00:06:44.383375 13320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:44.383689 13320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:44.384873 13320 hybrid_clock.cc:648] HybridClock initialized: now 1786493204384867 us; error 98 us; skew 500 ppm
Aug 12 00:06:44 dist-test-slave-z8x9 krb5kdc[13293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493204, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:44.391980 13320 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:44.392500 13320 webserver.cc:460] Webserver started at http://127.7.85.254:40199/ using document root <none> and password file <none>
I20260812 00:06:44.392771 13320 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:44.392843 13320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:44.392989 13320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:44.394169 13320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "fbb9179e8cfa49e5a9845ce95eb278cb"
format_stamp: "Formatted at 2026-08-12 00:06:44 on dist-test-slave-z8x9"
server_key: "457c7cc93d0a64434df963b2e370c777"
server_key_iv: "68f96b278fdd6ce7ed5558d72620d075"
server_key_version: "encryptionkey@0"
I20260812 00:06:44.394557 13320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "fbb9179e8cfa49e5a9845ce95eb278cb"
format_stamp: "Formatted at 2026-08-12 00:06:44 on dist-test-slave-z8x9"
server_key: "457c7cc93d0a64434df963b2e370c777"
server_key_iv: "68f96b278fdd6ce7ed5558d72620d075"
server_key_version: "encryptionkey@0"
I20260812 00:06:44.396049 13320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:44.397051 13355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:44.397406 13320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:44.397730 13320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "fbb9179e8cfa49e5a9845ce95eb278cb"
format_stamp: "Formatted at 2026-08-12 00:06:44 on dist-test-slave-z8x9"
server_key: "457c7cc93d0a64434df963b2e370c777"
server_key_iv: "68f96b278fdd6ce7ed5558d72620d075"
server_key_version: "encryptionkey@0"
I20260812 00:06:44.397837 13320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:44.408301 13320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:44.411365 13320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:44.411563 13320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:44.415972 13320 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:40239
I20260812 00:06:44.416402 13320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:44.416364 13463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:40239 every 8 connection(s)
I20260812 00:06:44.417361 13466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:44.420228 13466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb: Bootstrap starting.
I20260812 00:06:44.421144 13466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:44.421561 13466 log.cc:824] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:44.422451 13466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb: No bootstrap required, opened a new log
I20260812 00:06:44.422544 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13320
I20260812 00:06:44.422657 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:44.422878 7511 external_mini_cluster.cc:1428] Setting key 6f5656e317204e6967d34998c95aed5d
I20260812 00:06:44.425511 13466 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40239 } }
I20260812 00:06:44.425683 13466 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:44.425751 13466 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb9179e8cfa49e5a9845ce95eb278cb, State: Initialized, Role: FOLLOWER
I20260812 00:06:44.425923 13466 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40239 } }
I20260812 00:06:44.426025 13466 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:44.426072 13466 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:44.426127 13466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:44.426752 13466 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40239 } }
I20260812 00:06:44.426882 13466 leader_election.cc:302] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbb9179e8cfa49e5a9845ce95eb278cb; no voters:
I20260812 00:06:44.427098 13466 leader_election.cc:288] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:44.427131 13477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:44.427254 13477 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [term 1 LEADER]: Becoming Leader. State: Replica: fbb9179e8cfa49e5a9845ce95eb278cb, State: Running, Role: LEADER
I20260812 00:06:44.427402 13466 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:44.427402 13477 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40239 } }
I20260812 00:06:44.429112 13481 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40239 } } }
I20260812 00:06:44.429136 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbb9179e8cfa49e5a9845ce95eb278cb. Latest consensus state: current_term: 1 leader_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb9179e8cfa49e5a9845ce95eb278cb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40239 } } }
I20260812 00:06:44.429237 13481 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:44.429247 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb [sys.catalog]: This master's current role is: LEADER
W20260812 00:06:44.429663 13511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:06:44.429759 13511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:06:44.429836 13512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:44.430068 13512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 00:06:44 dist-test-slave-z8x9 krb5kdc[13293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:44.431937 13512 catalog_manager.cc:1347] Generated new cluster ID: 40b5f5e30e504ceaa8bc98c9c30c27a2
I20260812 00:06:44.432135 13512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:44.437028 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:44.423516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48114 (local address 127.7.85.254:40239)
0812 00:06:44.423736 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:44.423825 (+ 89us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:44.423866 (+ 41us) server_negotiation.cc:407] Connection header received
0812 00:06:44.425492 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:44.425501 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:44.425561 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:44.425730 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:44.426520 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:44.427761 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:44.428809 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:44.429494 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:44.429618 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:44.432536 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:44.432540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:44.432541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:44.432554 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:44.434723 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:44.435310 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:44.435313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:44.435314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:44.435443 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:44.435679 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:44.435679 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:44.435680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:44.436011 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:44.436159 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:44.436646 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 00:06:44.436804 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260812 00:06:44.438352 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:06:44.444041 13512 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:44.444811 13512 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:44.453053 13512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbb9179e8cfa49e5a9845ce95eb278cb: Generated new TSK 0
I20260812 00:06:44.453315 13512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:06:44.459656 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:44.454980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.7.85.254:40239)
0812 00:06:44.455047 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:44.455050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:44.455135 (+ 85us) server_negotiation.cc:407] Connection header received
0812 00:06:44.455201 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:44.455202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:44.455216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:44.455306 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:44.455772 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:44.456669 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:44.457663 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:44.457897 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:44.457934 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:44.458843 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:44.458846 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:44.458847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:44.459323 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:44.459344 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:44.459353 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:44.459430 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:44.459526 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 00:06:44.459564 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 00:06:44.462965 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13320
2026-08-12T00:06:44Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1686 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:44 dist-test-slave-z8x9 krb5kdc[13527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:44 dist-test-slave-z8x9 krb5kdc[13527](info): set up 2 sockets
Aug 12 00:06:44 dist-test-slave-z8x9 krb5kdc[13527](info): commencing operation
krb5kdc: starting...
W20260812 00:06:45.822819 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:45 dist-test-slave-z8x9 krb5kdc[13527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:45.965885 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39895
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:46.052834 13558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:46.053089 13558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:46.053129 13558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:46.054286 13558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:46.054363 13558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:46.054388 13558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:46.054417 13558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:46.054440 13558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:46.055979 13558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:39895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:46.056273 13558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:46.056526 13558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:46.061924 13574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.061924 13573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.061972 13576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:46.062615 13558 server_base.cc:1034] running on GCE node
I20260812 00:06:46.062927 13558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:46.063198 13558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:46.064371 13558 hybrid_clock.cc:648] HybridClock initialized: now 1786493206064355 us; error 29 us; skew 500 ppm
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:46.067215 13558 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:46.067763 13558 webserver.cc:460] Webserver started at http://127.7.85.254:45491/ using document root <none> and password file <none>
I20260812 00:06:46.068028 13558 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:46.068102 13558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:46.068281 13558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:46.069818 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "2bb5b30266acec2fa0c4aa1e28909fee"
server_key_iv: "f3081c423843bdd80fca11f7518ceb97"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.070623 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "2bb5b30266acec2fa0c4aa1e28909fee"
server_key_iv: "f3081c423843bdd80fca11f7518ceb97"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.072285 13558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:46.073319 13604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.073544 13558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:46.073657 13558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "2bb5b30266acec2fa0c4aa1e28909fee"
server_key_iv: "f3081c423843bdd80fca11f7518ceb97"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.073750 13558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:46.091490 13558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:46.094396 13558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:46.094589 13558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:46.100580 13558 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:39895
I20260812 00:06:46.100819 13713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:39895 every 8 connection(s)
I20260812 00:06:46.101068 13558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:46.102140 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13558
I20260812 00:06:46.102324 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:46.102540 7511 external_mini_cluster.cc:1428] Setting key 019f99284c86c6058aee803402bab5c4
I20260812 00:06:46.102949 13716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:46.106473 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60: Bootstrap starting.
I20260812 00:06:46.107124 13716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:46.107484 13716 log.cc:824] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:46.108342 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60: No bootstrap required, opened a new log
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:46.110710 13716 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39895 } }
I20260812 00:06:46.110906 13716 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:46.110934 13716 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b4acfe9f774e58bbed5bf9fe9c4c60, State: Initialized, Role: FOLLOWER
I20260812 00:06:46.111083 13716 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39895 } }
I20260812 00:06:46.111196 13716 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:46.111238 13716 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:46.111316 13716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:46.112089 13716 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39895 } }
I20260812 00:06:46.112221 13716 leader_election.cc:302] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0b4acfe9f774e58bbed5bf9fe9c4c60; no voters:
I20260812 00:06:46.112418 13716 leader_election.cc:288] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:46.112468 13727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:46.112674 13727 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [term 1 LEADER]: Becoming Leader. State: Replica: c0b4acfe9f774e58bbed5bf9fe9c4c60, State: Running, Role: LEADER
I20260812 00:06:46.112758 13716 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:46.112823 13727 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39895 } }
I20260812 00:06:46.113222 13727 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0b4acfe9f774e58bbed5bf9fe9c4c60. Latest consensus state: current_term: 1 leader_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39895 } } }
I20260812 00:06:46.113356 13728 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4acfe9f774e58bbed5bf9fe9c4c60" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39895 } } }
I20260812 00:06:46.113359 13727 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:46.113426 13728 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:46.113703 13746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:46.114383 13746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:46.115687 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.103171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.7.85.254:39895)
0812 00:06:46.103394 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:46.103404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:46.103438 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:06:46.104164 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:46.104174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:46.104241 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:46.104409 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:46.105092 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.106511 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.107419 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.107991 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.108107 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.111242 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:46.111247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:46.111248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:46.111263 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:46.113680 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.114199 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.114202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.114203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.114330 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.114567 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.114568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.114568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.114996 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:46.115151 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:46.115334 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 00:06:46.115480 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
I20260812 00:06:46.116289 13746 catalog_manager.cc:1347] Generated new cluster ID: 6e264055d8d24aa2b4ed466c4f99582c
I20260812 00:06:46.116343 13746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:46.147943 13746 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:46.148638 13746 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:46.156107 13746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0b4acfe9f774e58bbed5bf9fe9c4c60: Generated new TSK 0
I20260812 00:06:46.156345 13746 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:46.206255 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39895
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:46.296151 13774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:46.296393 13774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:46.296422 13774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:46.298379 13774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:46.298497 13774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:46.298537 13774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:46.301251 13774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:46.301582 13774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:46.301869 13774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:46.307639 13782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.307641 13787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.307679 13785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:46.308012 13774 server_base.cc:1034] running on GCE node
I20260812 00:06:46.308235 13774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:46.308502 13774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:46.309698 13774 hybrid_clock.cc:648] HybridClock initialized: now 1786493206309682 us; error 34 us; skew 500 ppm
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:46.312702 13774 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:46.313297 13774 webserver.cc:460] Webserver started at http://127.7.85.193:37617/ using document root <none> and password file <none>
I20260812 00:06:46.313532 13774 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:46.313578 13774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:46.313706 13774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:46.314823 13774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "09c0489173e14fa585e553cff3c0d358"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "5bd146b9c08f7c522ea8b425dbddae3c"
server_key_iv: "4afd1714e09f00f1d2e39c4c854ea3b7"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.315145 13774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "09c0489173e14fa585e553cff3c0d358"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "5bd146b9c08f7c522ea8b425dbddae3c"
server_key_iv: "4afd1714e09f00f1d2e39c4c854ea3b7"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.317616 13774 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 00:06:46.318759 13802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.319100 13774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:46.319211 13774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "09c0489173e14fa585e553cff3c0d358"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "5bd146b9c08f7c522ea8b425dbddae3c"
server_key_iv: "4afd1714e09f00f1d2e39c4c854ea3b7"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.319293 13774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:46.351996 13774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:46.355123 13774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:46.355288 13774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:46.355559 13774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:46.356208 13774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:46.356307 13774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.356338 13774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:46.356351 13774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.364253 13774 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:33467
I20260812 00:06:46.364399 14043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:33467 every 8 connection(s)
I20260812 00:06:46.364743 13774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:46.373853 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13774
I20260812 00:06:46.373979 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:46.374192 7511 external_mini_cluster.cc:1428] Setting key 71fb6c93eaa5567804829e0ff1f78416
I20260812 00:06:46.375947 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.365718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:55261 (local address 127.7.85.254:39895)
0812 00:06:46.365834 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:46.365837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:46.366028 (+ 191us) server_negotiation.cc:407] Connection header received
0812 00:06:46.366593 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:46.366594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:46.366608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:46.366669 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:46.367770 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.368295 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.369372 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.369578 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.369668 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.372436 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:46.372439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:46.372439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:46.372447 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:46.374358 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.374808 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.374809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.374810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.374877 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.375169 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.375171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.375171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.375335 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:46.375409 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:46.375700 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 00:06:46.375801 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:06:46.376132 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.365600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39895 (local address 127.7.85.193:55261)
0812 00:06:46.365872 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:46.365926 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:46.366476 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:46.366741 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:46.366746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:46.367414 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:46.367648 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:46.367653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.368395 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.368397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:46.369273 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:46.369275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.369680 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.369681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:46.369815 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.370653 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:46.370662 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:46.372295 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:46.374481 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:46.374489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:46.374491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:46.374740 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:46.374970 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:46.374971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:46.374971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:46.375095 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:46.375414 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:46.375420 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:46.375555 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 00:06:46.375647 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 00:06:46.375803 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":118,"threads_started":1}
I20260812 00:06:46.376856 14044 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39895
I20260812 00:06:46.376974 14044 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:46.377239 14044 heartbeater.cc:511] Master 127.7.85.254:39895 requested a full tablet report, sending...
I20260812 00:06:46.377918 13646 ts_manager.cc:194] Registered new tserver with Master: 09c0489173e14fa585e553cff3c0d358 (127.7.85.193:33467)
I20260812 00:06:46.378798 13646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:55261
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:46.413352 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39895
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:46.508987 14053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:46.509307 14053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:46.509339 14053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:46.510532 14053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:46.510578 14053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:46.510605 14053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:46.512113 14053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:46.512465 14053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:46.512725 14053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:46.519410 14060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.519444 14066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.519667 14062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:46.519905 14053 server_base.cc:1034] running on GCE node
I20260812 00:06:46.520107 14053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:46.520363 14053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:46.521555 14053 hybrid_clock.cc:648] HybridClock initialized: now 1786493206521527 us; error 43 us; skew 500 ppm
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:46.525321 14053 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:46.525847 14053 webserver.cc:460] Webserver started at http://127.7.85.194:38353/ using document root <none> and password file <none>
I20260812 00:06:46.526130 14053 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:46.526202 14053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:46.526377 14053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:46.527482 14053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "5cb55b2f1d604407bb394e3d5a482ad1"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "a10a5fe8a30924720ebe0edabe363b72"
server_key_iv: "1b31560a1710c31df083de39f95ecad9"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.527863 14053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "5cb55b2f1d604407bb394e3d5a482ad1"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "a10a5fe8a30924720ebe0edabe363b72"
server_key_iv: "1b31560a1710c31df083de39f95ecad9"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.529430 14053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:46.530287 14077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.530493 14053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:46.530606 14053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "5cb55b2f1d604407bb394e3d5a482ad1"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "a10a5fe8a30924720ebe0edabe363b72"
server_key_iv: "1b31560a1710c31df083de39f95ecad9"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.530685 14053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:46.550257 14053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:46.552956 14053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:46.553156 14053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:46.553488 14053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:46.553831 14053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:46.553898 14053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.553965 14053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:46.553998 14053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.561450 14053 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:36119
I20260812 00:06:46.561498 14305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:36119 every 8 connection(s)
I20260812 00:06:46.561880 14053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:46.570777 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14053
I20260812 00:06:46.570921 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:46.571141 7511 external_mini_cluster.cc:1428] Setting key 8b2075c289230e58249424f0941c1158
I20260812 00:06:46.572598 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.562778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:45951 (local address 127.7.85.254:39895)
0812 00:06:46.562845 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:46.562847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:46.563111 (+ 264us) server_negotiation.cc:407] Connection header received
0812 00:06:46.563687 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:46.563689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:46.563702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:46.563758 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:46.564790 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.565356 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.566390 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.566586 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.566680 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.569132 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:46.569134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:46.569134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:46.569143 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:46.570696 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.571250 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.571252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.571253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.571339 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.571684 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.571685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.571686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.571837 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:46.571911 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:46.572432 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 00:06:46.572529 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 00:06:46.572901 14312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.562760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39895 (local address 127.7.85.194:45951)
0812 00:06:46.562978 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:46.563018 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:46.563572 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:46.563830 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:46.563834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:46.564453 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:46.564674 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:46.564679 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.565457 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.565458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:46.566279 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:46.566282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.566694 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.566695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:46.566832 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.567418 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:46.567427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:46.569023 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:46.570838 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:46.570846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:46.570849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:46.571149 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:46.571421 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:46.571422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:46.571422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:46.571597 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:46.572024 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:46.572033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:46.572253 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 00:06:46.572391 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 00:06:46.572602 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260812 00:06:46.573419 14307 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39895
I20260812 00:06:46.573522 14307 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:46.573736 14307 heartbeater.cc:511] Master 127.7.85.254:39895 requested a full tablet report, sending...
I20260812 00:06:46.574155 13646 ts_manager.cc:194] Registered new tserver with Master: 5cb55b2f1d604407bb394e3d5a482ad1 (127.7.85.194:36119)
I20260812 00:06:46.574795 13646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:45951
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:46.605989 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39895
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:46.700673 14316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:46.700933 14316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:46.700963 14316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:46.702370 14316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:46.702425 14316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:46.702453 14316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:46.703999 14316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:46.704265 14316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:46.704545 14316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:46.709965 14322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.709973 14324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:46.709973 14328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:46.710160 14316 server_base.cc:1034] running on GCE node
I20260812 00:06:46.710458 14316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:46.710676 14316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:46.711850 14316 hybrid_clock.cc:648] HybridClock initialized: now 1786493206711839 us; error 32 us; skew 500 ppm
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:46.715531 14316 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:46.716116 14316 webserver.cc:460] Webserver started at http://127.7.85.195:43127/ using document root <none> and password file <none>
I20260812 00:06:46.716361 14316 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:46.716431 14316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:46.716591 14316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:46.717765 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "2007b8ab2de44998a5c7864390b8aacb"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "99cbe10bc7dc6b04e2e02980ae9945fd"
server_key_iv: "61a9047667949543fb0059c5c09e1e74"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.718139 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "2007b8ab2de44998a5c7864390b8aacb"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "99cbe10bc7dc6b04e2e02980ae9945fd"
server_key_iv: "61a9047667949543fb0059c5c09e1e74"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.719554 14316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:46.720453 14343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.720677 14316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:46.720788 14316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "2007b8ab2de44998a5c7864390b8aacb"
format_stamp: "Formatted at 2026-08-12 00:06:46 on dist-test-slave-z8x9"
server_key: "99cbe10bc7dc6b04e2e02980ae9945fd"
server_key_iv: "61a9047667949543fb0059c5c09e1e74"
server_key_version: "encryptionkey@0"
I20260812 00:06:46.720877 14316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:46.756121 14316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:46.759234 14316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:46.759487 14316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:46.759770 14316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:46.760131 14316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:46.760190 14316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.760232 14316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:46.760263 14316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:46.767198 14316 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34673
I20260812 00:06:46.767246 14560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34673 every 8 connection(s)
I20260812 00:06:46.767617 14316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:46.774147 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14316
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:46.774271 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:46.774475 7511 external_mini_cluster.cc:1428] Setting key b3e1cb21edf6412ec8ca03aa84b36fd7
I20260812 00:06:46.778322 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.768469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39895 (local address 127.7.85.195:43441)
0812 00:06:46.768652 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:46.768698 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:46.769222 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:46.769484 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:46.769487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:46.770107 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:46.770310 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:46.770314 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.771088 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.771090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:46.771943 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:46.771947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.772337 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.772339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:46.772470 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.773047 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:46.773057 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:46.774716 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:46.776483 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:46.776490 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:46.776492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:46.776850 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:46.777121 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:46.777122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:46.777122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:46.777351 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:46.777672 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:46.777679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:46.777855 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 00:06:46.777945 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:06:46.778087 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260812 00:06:46.778559 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.768488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:43441 (local address 127.7.85.254:39895)
0812 00:06:46.768560 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:46.768562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:46.768800 (+ 238us) server_negotiation.cc:407] Connection header received
0812 00:06:46.769323 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:46.769324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:46.769335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:46.769402 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:46.770437 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.770984 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.772040 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.772235 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.772300 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.774824 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:46.774825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:46.774825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:46.774834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:46.776376 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.776934 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.776935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.776935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.777034 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.777449 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.777450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.777450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.777585 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:46.777677 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:46.778194 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 00:06:46.778349 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:06:46.779153 14562 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39895
I20260812 00:06:46.779300 14562 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:46.779500 14562 heartbeater.cc:511] Master 127.7.85.254:39895 requested a full tablet report, sending...
I20260812 00:06:46.779951 13646 ts_manager.cc:194] Registered new tserver with Master: 2007b8ab2de44998a5c7864390b8aacb (127.7.85.195:34673)
I20260812 00:06:46.780362 13646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:43441
I20260812 00:06:46.786923 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[13527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493206, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:46.813596 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:46.805806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.7.85.254:39895)
0812 00:06:46.805911 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:46.805915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:46.806029 (+ 114us) server_negotiation.cc:407] Connection header received
0812 00:06:46.806126 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:46.806127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:46.806144 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:46.806205 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:46.806857 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.807370 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.808130 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:46.808325 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:46.808394 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:46.810928 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:46.810930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:46.810930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:46.810938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:46.812300 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.812642 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.812643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.812643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.812705 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:46.812923 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:46.812924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:46.812924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:46.813110 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:46.813236 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:46.813390 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 00:06:46.813481 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 00:06:46.813886 13646 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42292
I20260812 00:06:46.815038 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13774
I20260812 00:06:46.838972 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14053
I20260812 00:06:46.854167 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14316
I20260812 00:06:46.868337 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13558
2026-08-12T00:06:46Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2414 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[14588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[14588](info): set up 2 sockets
Aug 12 00:06:46 dist-test-slave-z8x9 krb5kdc[14588](info): commencing operation
krb5kdc: starting...
W20260812 00:06:48.231225 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:48.382478 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41037
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:48.474327 14611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:48.474618 14611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:48.474663 14611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:48.475991 14611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:48.476071 14611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:48.476099 14611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:48.476132 14611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:48.476158 14611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:48.478322 14611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:48.478717 14611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:48.478995 14611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:48.485546 14622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:48.485546 14625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:48.485771 14619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:48.485800 14611 server_base.cc:1034] running on GCE node
I20260812 00:06:48.486188 14611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:48.486583 14611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:48.487780 14611 hybrid_clock.cc:648] HybridClock initialized: now 1786493208487742 us; error 58 us; skew 500 ppm
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:48.490908 14611 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:48.491441 14611 webserver.cc:460] Webserver started at http://127.7.85.254:41897/ using document root <none> and password file <none>
I20260812 00:06:48.491765 14611 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:48.491839 14611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:48.492004 14611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:48.493158 14611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "7099ac8e353a48ae911c3e61e940ce3e"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "6ba296183350c081e45387e92ff2a206"
server_key_iv: "d08a7669ca836b5a3701a13f7048ee2e"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.493628 14611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "7099ac8e353a48ae911c3e61e940ce3e"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "6ba296183350c081e45387e92ff2a206"
server_key_iv: "d08a7669ca836b5a3701a13f7048ee2e"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.495277 14611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:48.496268 14643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.496654 14611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:48.496907 14611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "7099ac8e353a48ae911c3e61e940ce3e"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "6ba296183350c081e45387e92ff2a206"
server_key_iv: "d08a7669ca836b5a3701a13f7048ee2e"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.496994 14611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:48.507335 14611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:48.510561 14611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:48.510849 14611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:48.516124 14611 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:41037
I20260812 00:06:48.516378 14743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:41037 every 8 connection(s)
I20260812 00:06:48.517308 14611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:48.518409 14747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:48.519604 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14611
I20260812 00:06:48.519847 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:48.520058 7511 external_mini_cluster.cc:1428] Setting key 4188bc32197aeaabce79adc305d8882c
I20260812 00:06:48.521282 14747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e: Bootstrap starting.
I20260812 00:06:48.521981 14747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:48.522300 14747 log.cc:824] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:48.523059 14747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e: No bootstrap required, opened a new log
I20260812 00:06:48.524963 14747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7099ac8e353a48ae911c3e61e940ce3e" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41037 } }
I20260812 00:06:48.525151 14747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:48.525230 14747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7099ac8e353a48ae911c3e61e940ce3e, State: Initialized, Role: FOLLOWER
I20260812 00:06:48.525398 14747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7099ac8e353a48ae911c3e61e940ce3e" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41037 } }
I20260812 00:06:48.525496 14747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:48.525538 14747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:48.525581 14747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:48.526194 14747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7099ac8e353a48ae911c3e61e940ce3e" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41037 } }
I20260812 00:06:48.526324 14747 leader_election.cc:302] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7099ac8e353a48ae911c3e61e940ce3e; no voters:
I20260812 00:06:48.526556 14747 leader_election.cc:288] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:48.526589 14754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:48.526718 14754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [term 1 LEADER]: Becoming Leader. State: Replica: 7099ac8e353a48ae911c3e61e940ce3e, State: Running, Role: LEADER
I20260812 00:06:48.526823 14754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7099ac8e353a48ae911c3e61e940ce3e" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41037 } }
I20260812 00:06:48.526861 14747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:48.527179 14754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7099ac8e353a48ae911c3e61e940ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7099ac8e353a48ae911c3e61e940ce3e" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41037 } } }
I20260812 00:06:48.527249 14754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:48.527379 14754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7099ac8e353a48ae911c3e61e940ce3e. Latest consensus state: current_term: 1 leader_uuid: "7099ac8e353a48ae911c3e61e940ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7099ac8e353a48ae911c3e61e940ce3e" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41037 } } }
I20260812 00:06:48.527446 14754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:48.527590 14761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:48.528175 14761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:48.530241 14761 catalog_manager.cc:1347] Generated new cluster ID: afd0f75160d24e7a8ef14b3d6b50ff23
I20260812 00:06:48.530299 14761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:48.534457 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:48.520644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.7.85.254:41037)
0812 00:06:48.520931 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:48.520940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:48.520968 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:06:48.521625 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:48.521635 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:48.521689 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:48.521848 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:48.522687 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.523675 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.524478 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.524922 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.525241 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:48.530663 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:48.530667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:48.530668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:48.530690 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:48.532703 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:48.533164 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:48.533190 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:48.533190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:48.533300 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:48.533536 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:48.533537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:48.533538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:48.533811 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:48.534027 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:48.534110 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 00:06:48.534263 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":120,"threads_started":1}
I20260812 00:06:48.544399 14761 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:48.545384 14761 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:48.553884 14761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7099ac8e353a48ae911c3e61e940ce3e: Generated new TSK 0
I20260812 00:06:48.554147 14761 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:48.607483 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41037
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:48.709162 14803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:48.709529 14803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:48.709561 14803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:48.711063 14803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:48.711112 14803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:48.711140 14803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:48.713074 14803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:48.713532 14803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:48.713901 14803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:48.720029 14813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:48.719995 14821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:48.720124 14815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:48.720256 14803 server_base.cc:1034] running on GCE node
I20260812 00:06:48.720472 14803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:48.720758 14803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:48.721990 14803 hybrid_clock.cc:648] HybridClock initialized: now 1786493208721954 us; error 65 us; skew 500 ppm
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:48.725032 14803 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:48.725592 14803 webserver.cc:460] Webserver started at http://127.7.85.193:38689/ using document root <none> and password file <none>
I20260812 00:06:48.725795 14803 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:48.725837 14803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:48.725950 14803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:48.727105 14803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "222842424137411c994d36326097fdbd"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "66a217da8fe4fcfd9a1a56c428a02cdc"
server_key_iv: "0ddda05ad9053d009ef3351844ecfb62"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.727442 14803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "222842424137411c994d36326097fdbd"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "66a217da8fe4fcfd9a1a56c428a02cdc"
server_key_iv: "0ddda05ad9053d009ef3351844ecfb62"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.729153 14803 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:06:48.730150 14836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.730484 14803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:48.730581 14803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "222842424137411c994d36326097fdbd"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "66a217da8fe4fcfd9a1a56c428a02cdc"
server_key_iv: "0ddda05ad9053d009ef3351844ecfb62"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.730647 14803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:48.755863 14803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:48.758464 14803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:48.758602 14803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:48.758821 14803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:48.759176 14803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:48.759213 14803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.759239 14803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:48.759251 14803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.766415 14803 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:37089
I20260812 00:06:48.766456 15090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:37089 every 8 connection(s)
I20260812 00:06:48.766876 14803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:48.775207 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14803
I20260812 00:06:48.775347 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:48.775585 7511 external_mini_cluster.cc:1428] Setting key 4c883df0a5ced6d7b0307cee028a06f6
I20260812 00:06:48.778870 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:48.767897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:49883 (local address 127.7.85.254:41037)
0812 00:06:48.767989 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:48.767992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:48.768171 (+ 179us) server_negotiation.cc:407] Connection header received
0812 00:06:48.769262 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:48.769264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:48.769277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:48.769375 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:48.770083 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.770650 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.771842 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.772068 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.772146 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:48.775304 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:48.775307 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:48.775308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:48.775320 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:48.777308 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:48.777797 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:48.777798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:48.777799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:48.777870 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:48.778153 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:48.778154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:48.778154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:48.778290 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:48.778370 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:48.778668 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 00:06:48.778772 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 00:06:48.779104 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:48.767838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41037 (local address 127.7.85.193:49883)
0812 00:06:48.768037 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:48.768079 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:48.769139 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:48.769454 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:48.769458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:48.769751 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:48.769957 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:48.769960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.770752 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.770753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:48.771718 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:48.771721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.772214 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.772216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:48.772336 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:48.773032 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:48.773043 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:48.775173 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:48.777418 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:48.777428 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:48.777431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:48.777712 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:48.777952 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:48.777953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:48.777953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:48.778075 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:48.778361 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:48.778369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:48.778526 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 00:06:48.778646 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 00:06:48.778834 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260812 00:06:48.779734 15091 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41037
I20260812 00:06:48.779860 15091 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:48.780073 15091 heartbeater.cc:511] Master 127.7.85.254:41037 requested a full tablet report, sending...
I20260812 00:06:48.780764 14672 ts_manager.cc:194] Registered new tserver with Master: 222842424137411c994d36326097fdbd (127.7.85.193:37089)
I20260812 00:06:48.781826 14672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:49883
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:48.810607 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41037
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:48.900918 15099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:48.901211 15099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:48.901257 15099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:48.902474 15099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:48.902520 15099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:48.902549 15099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:48.904132 15099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:48.904425 15099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:48.904688 15099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:48.910488 15115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:48.910481 15111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:48.910481 15112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:48.910519 15099 server_base.cc:1034] running on GCE node
I20260812 00:06:48.910950 15099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:48.911252 15099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:48.912462 15099 hybrid_clock.cc:648] HybridClock initialized: now 1786493208912434 us; error 28 us; skew 500 ppm
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:48.915388 15099 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:48.916111 15099 webserver.cc:460] Webserver started at http://127.7.85.194:33985/ using document root <none> and password file <none>
I20260812 00:06:48.916428 15099 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:48.916530 15099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:48.916968 15099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:48.918200 15099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "44d193df92f14bcc988955fb037986a2"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "7d6d28a979cddc80f59d3406814d2f7f"
server_key_iv: "1db19df26c72bc3aaf4e4062d10591d7"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.918576 15099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "44d193df92f14bcc988955fb037986a2"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "7d6d28a979cddc80f59d3406814d2f7f"
server_key_iv: "1db19df26c72bc3aaf4e4062d10591d7"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.920065 15099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:48.921343 15134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.921540 15099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:48.921645 15099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "44d193df92f14bcc988955fb037986a2"
format_stamp: "Formatted at 2026-08-12 00:06:48 on dist-test-slave-z8x9"
server_key: "7d6d28a979cddc80f59d3406814d2f7f"
server_key_iv: "1db19df26c72bc3aaf4e4062d10591d7"
server_key_version: "encryptionkey@0"
I20260812 00:06:48.921703 15099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:48.948508 15099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:48.951067 15099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:48.951191 15099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:48.951411 15099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:48.951764 15099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:48.951805 15099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.951830 15099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:48.951843 15099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:48.958400 15099 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:37339
I20260812 00:06:48.958451 15355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:37339 every 8 connection(s)
I20260812 00:06:48.958802 15099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:06:48 dist-test-slave-z8x9 krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:48.967517 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15099
I20260812 00:06:48.967708 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:48.967932 7511 external_mini_cluster.cc:1428] Setting key 5747028353e7f6aadfb71e2cab670555
I20260812 00:06:48.970587 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:48.959652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:42875 (local address 127.7.85.254:41037)
0812 00:06:48.959739 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:48.959741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:48.959951 (+ 210us) server_negotiation.cc:407] Connection header received
0812 00:06:48.960934 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:48.960936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:48.960955 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:48.961035 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:48.961941 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.962773 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.964250 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.964486 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.964557 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:48.967211 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:48.967213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:48.967214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:48.967222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:48.968918 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:48.969529 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:48.969530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:48.969531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:48.969599 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:48.969875 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:48.969876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:48.969877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:48.970006 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:48.970079 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:48.970400 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 00:06:48.970490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:06:48.970801 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:48.959617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41037 (local address 127.7.85.194:42875)
0812 00:06:48.959793 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:48.959839 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:48.960811 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:48.961242 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:48.961248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:48.961578 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:48.961808 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:48.961813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.963019 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.963022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:48.964124 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:48.964131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:48.964601 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:48.964602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:48.964743 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:48.965448 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:48.965458 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:48.967098 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:48.969007 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:48.969016 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:48.969019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:48.969428 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:48.969668 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:48.969668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:48.969669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:48.969796 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:48.970080 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:48.970088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:48.970245 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 00:06:48.970354 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 00:06:48.970535 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260812 00:06:48.971644 15356 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41037
I20260812 00:06:48.971772 15356 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:48.971987 15356 heartbeater.cc:511] Master 127.7.85.254:41037 requested a full tablet report, sending...
I20260812 00:06:48.972532 14672 ts_manager.cc:194] Registered new tserver with Master: 44d193df92f14bcc988955fb037986a2 (127.7.85.194:37339)
I20260812 00:06:48.972965 14672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:42875
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:49.002082 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41037
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:49.092398 15370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:49.092636 15370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:49.092664 15370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:49.093972 15370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:49.094031 15370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:49.094061 15370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:49.095664 15370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:49.095927 15370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:49.096189 15370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:49.101653 15381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:49.101662 15386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:49.101653 15380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:49.102120 15370 server_base.cc:1034] running on GCE node
I20260812 00:06:49.102391 15370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:49.102655 15370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:49.103905 15370 hybrid_clock.cc:648] HybridClock initialized: now 1786493209103853 us; error 34 us; skew 500 ppm
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:49.107079 15370 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:49.107698 15370 webserver.cc:460] Webserver started at http://127.7.85.195:36115/ using document root <none> and password file <none>
I20260812 00:06:49.107954 15370 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:49.108031 15370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:49.108187 15370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:49.109411 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "540c298b4eac413984308f3843723ef9"
format_stamp: "Formatted at 2026-08-12 00:06:49 on dist-test-slave-z8x9"
server_key: "74f77e89e3461ce3d3d086f830b976e9"
server_key_iv: "5151c73b0ba8f7d9735f0617c7bd8ae6"
server_key_version: "encryptionkey@0"
I20260812 00:06:49.109783 15370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "540c298b4eac413984308f3843723ef9"
format_stamp: "Formatted at 2026-08-12 00:06:49 on dist-test-slave-z8x9"
server_key: "74f77e89e3461ce3d3d086f830b976e9"
server_key_iv: "5151c73b0ba8f7d9735f0617c7bd8ae6"
server_key_version: "encryptionkey@0"
I20260812 00:06:49.111270 15370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:49.112071 15397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:49.112345 15370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:49.112454 15370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "540c298b4eac413984308f3843723ef9"
format_stamp: "Formatted at 2026-08-12 00:06:49 on dist-test-slave-z8x9"
server_key: "74f77e89e3461ce3d3d086f830b976e9"
server_key_iv: "5151c73b0ba8f7d9735f0617c7bd8ae6"
server_key_version: "encryptionkey@0"
I20260812 00:06:49.112532 15370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:49.130513 15370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:49.133561 15370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:49.133754 15370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:49.134050 15370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:49.134478 15370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:49.134541 15370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:49.134579 15370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:49.134611 15370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:49.141278 15370 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:40581
I20260812 00:06:49.141341 15613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:40581 every 8 connection(s)
I20260812 00:06:49.141698 15370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:49.149708 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15370
I20260812 00:06:49.149830 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:49.150063 7511 external_mini_cluster.cc:1428] Setting key 5edd54a3c96c36c9f9faacd21a935cc3
I20260812 00:06:49.153600 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:49.142518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:40019 (local address 127.7.85.254:41037)
0812 00:06:49.142633 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:49.142635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:49.142822 (+ 187us) server_negotiation.cc:407] Connection header received
0812 00:06:49.143835 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:49.143837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:49.143852 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:49.143911 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:49.145112 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.145676 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.146833 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.147111 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.147298 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:49.150038 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:49.150040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:49.150040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:49.150050 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:49.151695 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:49.152216 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:49.152217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:49.152218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:49.152282 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:49.152858 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:49.152859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:49.152860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:49.152988 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:49.153054 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:49.153453 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 00:06:49.153530 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 00:06:49.153736 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:49.142433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41037 (local address 127.7.85.195:40019)
0812 00:06:49.142656 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:49.142710 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:49.143698 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:49.143988 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:49.143998 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:49.144459 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:49.144980 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:49.144987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.145786 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.145787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:49.146704 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:49.146707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.147310 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.147311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:49.147456 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:49.148052 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:49.148062 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:49.149920 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:49.151833 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:49.151842 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:49.151844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:49.152092 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:49.152392 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:49.152393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:49.152394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:49.152723 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:49.153057 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:49.153062 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:49.153264 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 00:06:49.153381 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 00:06:49.153530 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260812 00:06:49.154274 15615 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41037
I20260812 00:06:49.154423 15615 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:49.154649 15615 heartbeater.cc:511] Master 127.7.85.254:41037 requested a full tablet report, sending...
I20260812 00:06:49.155082 14672 ts_manager.cc:194] Registered new tserver with Master: 540c298b4eac413984308f3843723ef9 (127.7.85.195:40581)
I20260812 00:06:49.155481 14672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:40019
I20260812 00:06:49.162503 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[14588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:49.189859 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:49.180116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.7.85.193:37089)
0812 00:06:49.180446 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:49.180452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:49.180464 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:06:49.180493 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:49.180496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:49.180512 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:49.180590 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:49.181513 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.182319 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.183088 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.183431 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.183495 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:49.186494 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:49.186504 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:49.186504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:49.186514 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:49.188185 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:49.188592 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:49.188595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:49.188596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:49.188670 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:49.189118 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:49.189119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:49.189120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:49.189390 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:49.189514 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:49.189685 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 00:06:49.189767 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":255,"threads_started":1}
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[14588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:49.199978 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:49.191962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.7.85.254:41037)
0812 00:06:49.192043 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:49.192045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:49.192059 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:06:49.192081 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:49.192082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:49.192092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:49.192147 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:49.192744 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.193338 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.194055 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:49.194241 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:49.194304 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:49.196917 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:49.196919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:49.196919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:49.196927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:49.198425 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:49.198850 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:49.198851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:49.198851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:49.198920 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:49.199114 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:49.199115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:49.199116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:49.199321 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:49.199414 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:49.199693 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 00:06:49.199847 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 00:06:49.200346 14672 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39968
I20260812 00:06:49.200886 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14803
I20260812 00:06:49.226490 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15099
I20260812 00:06:49.239620 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15370
I20260812 00:06:49.257367 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14611
2026-08-12T00:06:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2384 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[15633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[15633](info): set up 2 sockets
Aug 12 00:06:49 dist-test-slave-z8x9 krb5kdc[15633](info): commencing operation
krb5kdc: starting...
W20260812 00:06:50.601867 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:50 dist-test-slave-z8x9 krb5kdc[15633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:50.758293 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40803
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:40803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:50.845963 15660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:50.846236 15660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:50.846263 15660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:50.847487 15660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:50.847540 15660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:50.847555 15660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:50.847568 15660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:50.847581 15660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:50.849259 15660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:40803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:50.849546 15660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:50.849823 15660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:50.855666 15667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:50.855700 15676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:50.855672 15672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:50.855875 15660 server_base.cc:1034] running on GCE node
I20260812 00:06:50.856258 15660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:50.856699 15660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:50.857934 15660 hybrid_clock.cc:648] HybridClock initialized: now 1786493210857857 us; error 126 us; skew 500 ppm
Aug 12 00:06:50 dist-test-slave-z8x9 krb5kdc[15633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493210, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:50.865240 15660 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:50.865698 15660 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:50.865777 15660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:50.865949 15660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:50.867105 15660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "271cd480983942a7b440c3171f532ee3"
format_stamp: "Formatted at 2026-08-12 00:06:50 on dist-test-slave-z8x9"
server_key: "14d38640c9a7a31e7e61932a61350778"
server_key_iv: "7459fbe925974f090c96282d5c48e1c9"
server_key_version: "encryptionkey@0"
I20260812 00:06:50.867491 15660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "271cd480983942a7b440c3171f532ee3"
format_stamp: "Formatted at 2026-08-12 00:06:50 on dist-test-slave-z8x9"
server_key: "14d38640c9a7a31e7e61932a61350778"
server_key_iv: "7459fbe925974f090c96282d5c48e1c9"
server_key_version: "encryptionkey@0"
I20260812 00:06:50.868978 15660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:50.870021 15682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:50.870289 15660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:50.870443 15660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "271cd480983942a7b440c3171f532ee3"
format_stamp: "Formatted at 2026-08-12 00:06:50 on dist-test-slave-z8x9"
server_key: "14d38640c9a7a31e7e61932a61350778"
server_key_iv: "7459fbe925974f090c96282d5c48e1c9"
server_key_version: "encryptionkey@0"
I20260812 00:06:50.870545 15660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:50.885962 15660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:50.888998 15660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:50.889320 15660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:50.894729 15660 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:40803
I20260812 00:06:50.894879 15777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:40803 every 8 connection(s)
I20260812 00:06:50.895152 15660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:50.896368 15778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:50.899127 15778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3: Bootstrap starting.
I20260812 00:06:50.899873 15778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:50.900221 15778 log.cc:824] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:50.901247 15778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3: No bootstrap required, opened a new log
I20260812 00:06:50.902989 15778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271cd480983942a7b440c3171f532ee3" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40803 } }
I20260812 00:06:50.903158 15778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:50.903183 15778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271cd480983942a7b440c3171f532ee3, State: Initialized, Role: FOLLOWER
I20260812 00:06:50.903353 15778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271cd480983942a7b440c3171f532ee3" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40803 } }
I20260812 00:06:50.903435 15778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:50.903494 15778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:50.903551 15778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:50.904532 15778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271cd480983942a7b440c3171f532ee3" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40803 } }
I20260812 00:06:50.904635 15778 leader_election.cc:302] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271cd480983942a7b440c3171f532ee3; no voters:
I20260812 00:06:50.904847 15778 leader_election.cc:288] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:50.904909 15783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:50.905143 15783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 271cd480983942a7b440c3171f532ee3, State: Running, Role: LEADER
I20260812 00:06:50.905195 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15660
I20260812 00:06:50.905306 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:50.905216 15778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:50.905522 7511 external_mini_cluster.cc:1428] Setting key 3ef9ac6ae38d8934544bb9004b1f2d52
I20260812 00:06:50.905546 15783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271cd480983942a7b440c3171f532ee3" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40803 } }
I20260812 00:06:50.905993 15785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "271cd480983942a7b440c3171f532ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271cd480983942a7b440c3171f532ee3" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40803 } } }
I20260812 00:06:50.906102 15785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:50.906013 15789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271cd480983942a7b440c3171f532ee3. Latest consensus state: current_term: 1 leader_uuid: "271cd480983942a7b440c3171f532ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271cd480983942a7b440c3171f532ee3" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40803 } } }
I20260812 00:06:50.906209 15789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:50.906365 15800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:50.908010 15800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:50.909969 15800 catalog_manager.cc:1347] Generated new cluster ID: c7dcca941f434fa68dcec999a6b40205
I20260812 00:06:50.910035 15800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:06:50 dist-test-slave-z8x9 krb5kdc[15633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:50.919221 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:50.906216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.7.85.254:40803)
0812 00:06:50.907061 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:50.907366 (+ 305us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:50.907433 (+ 67us) server_negotiation.cc:407] Connection header received
0812 00:06:50.907615 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:50.907629 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:50.907746 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:50.908017 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:50.909132 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:50.910479 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:50.911293 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:50.911729 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:50.911859 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:50.914411 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:50.914416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:50.914416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:50.914438 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:50.916582 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:50.917106 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:50.917110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:50.917111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:50.917309 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:50.917602 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:50.917603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:50.917603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:50.917936 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:50.918493 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:50.918625 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 00:06:50.918937 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":79,"threads_started":1}
I20260812 00:06:50.925839 15800 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:50.926476 15800 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:50.931090 15800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3: Generated new TSK 0
I20260812 00:06:50.931412 15800 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:50.978348 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40803
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:51.087210 15834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:51.087462 15834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:51.087491 15834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:51.089001 15834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:51.089079 15834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:51.089116 15834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:51.091046 15834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:51.091703 15834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:51.091913 15834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:51.098112 15847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:51.098178 15834 server_base.cc:1034] running on GCE node
W20260812 00:06:51.098584 15841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.098655 15842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:51.099004 15834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:51.099316 15834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:51.100517 15834 hybrid_clock.cc:648] HybridClock initialized: now 1786493211100476 us; error 57 us; skew 500 ppm
Aug 12 00:06:51 dist-test-slave-z8x9 krb5kdc[15633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:51.103919 15834 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:51.104337 15834 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:51.104419 15834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:51.104578 15834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:51.105829 15834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "f9fe443214da4fc68fd418789c111343"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "19be03708cbad16b873f04e60cea5521"
server_key_iv: "811353d686c36d40dbc5ca9ac69bf225"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.106213 15834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "f9fe443214da4fc68fd418789c111343"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "19be03708cbad16b873f04e60cea5521"
server_key_iv: "811353d686c36d40dbc5ca9ac69bf225"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.107741 15834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:51.108814 15855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.109063 15834 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:51.109175 15834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "f9fe443214da4fc68fd418789c111343"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "19be03708cbad16b873f04e60cea5521"
server_key_iv: "811353d686c36d40dbc5ca9ac69bf225"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.109330 15834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:51.119086 15834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:51.122946 15834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:51.123178 15834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:51.123456 15834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:51.123860 15834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:51.123919 15834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.123968 15834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:51.123998 15834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.130209 15834 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:37701
I20260812 00:06:51.130295 16077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:37701 every 8 connection(s)
I20260812 00:06:51.130599 15834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:51.138451 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15834
I20260812 00:06:51.138587 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:51.138798 7511 external_mini_cluster.cc:1428] Setting key 3394295aa690fb41ad152ecc26c07f0b
Aug 12 00:06:51 dist-test-slave-z8x9 krb5kdc[15633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:51.143862 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.131400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40803 (local address 127.7.85.193:41329)
0812 00:06:51.131739 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.131815 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.132509 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.132965 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.132970 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.133374 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:51.134056 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.134064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.135030 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.135031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.135950 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.135954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.136720 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.136722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.136878 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.137534 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:51.137553 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:51.139398 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:51.141601 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:51.141611 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:51.141614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:51.142335 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:51.142637 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:51.142638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:51.142639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:51.142824 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:51.143125 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:51.143133 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:51.143334 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.143475 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.143646 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
I20260812 00:06:51.143900 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.131400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:41329 (local address 127.7.85.254:40803)
0812 00:06:51.131470 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.131472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.131977 (+ 505us) server_negotiation.cc:407] Connection header received
0812 00:06:51.132774 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.132775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.132794 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.132887 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:51.134236 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.134863 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.136324 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.136573 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.136662 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.139602 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:51.139605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:51.139605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:51.139619 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:51.141499 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.142474 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.142476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.142477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.142553 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.142916 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.142917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.142917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.143051 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:51.143112 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.143731 (+ 619us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.143831 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 00:06:51.144526 16078 heartbeater.cc:347] Connected to a master server at 127.7.85.254:40803
I20260812 00:06:51.144665 16078 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:51.144848 16078 heartbeater.cc:511] Master 127.7.85.254:40803 requested a full tablet report, sending...
I20260812 00:06:51.145476 15712 ts_manager.cc:194] Registered new tserver with Master: f9fe443214da4fc68fd418789c111343 (127.7.85.193:37701)
I20260812 00:06:51.146204 15712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:41329
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:51.178330 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40803
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:51.274967 16088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:51.275266 16088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:51.275341 16088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:51.276552 16088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:51.276625 16088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:51.276672 16088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:51.278343 16088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:51.278636 16088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:51.278892 16088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:06:51.284521 16088 server_base.cc:1034] running on GCE node
W20260812 00:06:51.284504 16104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.284504 16100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.284503 16101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:51.285095 16088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:51.285440 16088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:51.286652 16088 hybrid_clock.cc:648] HybridClock initialized: now 1786493211286610 us; error 94 us; skew 500 ppm
Aug 12 00:06:51 dist-test-slave-z8x9 krb5kdc[15633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:51.290014 16088 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:51.290433 16088 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:51.290511 16088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:51.290661 16088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:51.291751 16088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "e3321e03dd6345f3b9ea1e81af383a4c"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "f785993439ba7fab4ae670b01006c579"
server_key_iv: "330ad732e89aeb8399bca55812547af9"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.292073 16088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "e3321e03dd6345f3b9ea1e81af383a4c"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "f785993439ba7fab4ae670b01006c579"
server_key_iv: "330ad732e89aeb8399bca55812547af9"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.293507 16088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:06:51.294355 16118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.294530 16088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:51.294623 16088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "e3321e03dd6345f3b9ea1e81af383a4c"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "f785993439ba7fab4ae670b01006c579"
server_key_iv: "330ad732e89aeb8399bca55812547af9"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.294697 16088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:51.308799 16088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:51.311290 16088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:51.311461 16088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:51.311738 16088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:51.312084 16088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:51.312145 16088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.312191 16088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:51.312218 16088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.318885 16088 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:37329
I20260812 00:06:51.318919 16344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:37329 every 8 connection(s)
I20260812 00:06:51.319231 16088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:06:51 dist-test-slave-z8x9 krb5kdc[15633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:51.326531 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16088
I20260812 00:06:51.326653 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:51.326869 7511 external_mini_cluster.cc:1428] Setting key ddafb31e1390558160cc5a9a3a2cef53
I20260812 00:06:51.330739 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.319984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:36293 (local address 127.7.85.254:40803)
0812 00:06:51.320074 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.320077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.320376 (+ 299us) server_negotiation.cc:407] Connection header received
0812 00:06:51.320911 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.320912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.320924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.320992 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:51.322110 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.322762 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.323903 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.324110 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.324181 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.326921 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:51.326923 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:51.326924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:51.326935 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:51.328861 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.329427 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.329428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.329429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.329501 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.329852 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.329853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.329854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.329999 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:51.330052 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.330461 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.330668 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 00:06:51.331064 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.319960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40803 (local address 127.7.85.194:36293)
0812 00:06:51.320240 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.320281 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.320795 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.321077 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.321081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.321479 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:51.321993 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.321998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.322919 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.322920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.323775 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.323778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.324223 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.324224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.324351 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.325038 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:51.325048 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:51.326811 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:51.328975 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:51.328984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:51.328987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:51.329345 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:51.329584 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:51.329584 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:51.329585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:51.329757 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:51.330068 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:51.330075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:51.330266 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.330388 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.330592 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":117,"threads_started":1}
I20260812 00:06:51.331688 16346 heartbeater.cc:347] Connected to a master server at 127.7.85.254:40803
I20260812 00:06:51.331806 16346 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:51.331982 16346 heartbeater.cc:511] Master 127.7.85.254:40803 requested a full tablet report, sending...
I20260812 00:06:51.332459 15712 ts_manager.cc:194] Registered new tserver with Master: e3321e03dd6345f3b9ea1e81af383a4c (127.7.85.194:37329)
I20260812 00:06:51.333132 15712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:36293
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:51.363158 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40803
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:51.474826 16356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:51.475163 16356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:51.475193 16356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:51.476588 16356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:51.476649 16356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:51.476680 16356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:51.478425 16356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:51.478714 16356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:51.478982 16356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:51.485445 16375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.485466 16370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.485481 16366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:51.485594 16356 server_base.cc:1034] running on GCE node
I20260812 00:06:51.486105 16356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:51.486382 16356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:51.487562 16356 hybrid_clock.cc:648] HybridClock initialized: now 1786493211487519 us; error 72 us; skew 500 ppm
Aug 12 00:06:51 dist-test-slave-z8x9 krb5kdc[15633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:51.490562 16356 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:51.491346 16356 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:51.491467 16356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:51.491680 16356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:51.493418 16356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "446b7cc413c14a7b86aa84cddf4856b0"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "d8f0b804735e501290900f71e7d13328"
server_key_iv: "a6c6fc7033c0cc9c2124859f6e8ac296"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.493834 16356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "446b7cc413c14a7b86aa84cddf4856b0"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "d8f0b804735e501290900f71e7d13328"
server_key_iv: "a6c6fc7033c0cc9c2124859f6e8ac296"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.495249 16356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:06:51.496153 16382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.496498 16356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:51.496681 16356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "446b7cc413c14a7b86aa84cddf4856b0"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "d8f0b804735e501290900f71e7d13328"
server_key_iv: "a6c6fc7033c0cc9c2124859f6e8ac296"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.496798 16356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:51.509258 16356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:51.511875 16356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:51.512046 16356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:51.512336 16356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:51.512758 16356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:51.512812 16356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.512849 16356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:51.512878 16356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:51.520155 16356 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:39895
I20260812 00:06:51.520202 16593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:39895 every 8 connection(s)
I20260812 00:06:51.520489 16356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:51.520709 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16356
I20260812 00:06:51.520823 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:51.521034 7511 external_mini_cluster.cc:1428] Setting key f2da922e59747a38baba255bcdfb1902
Aug 12 00:06:51 dist-test-slave-z8x9 krb5kdc[15633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:51.531538 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.521481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:54829 (local address 127.7.85.254:40803)
0812 00:06:51.521548 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.521550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.521878 (+ 328us) server_negotiation.cc:407] Connection header received
0812 00:06:51.522519 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.522520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.522533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.522596 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:51.523606 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.524130 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.525160 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.525402 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.525494 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.527944 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:51.527946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:51.527947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:51.527955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:51.529928 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.530426 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.530427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.530428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.530500 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.530775 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.530776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.530777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.530937 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:51.531016 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.531339 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.531432 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 00:06:51.531839 16597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.521509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40803 (local address 127.7.85.195:54829)
0812 00:06:51.521725 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.521778 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.522409 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.522668 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.522672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.522958 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:51.523479 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.523483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.524232 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.524233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.525058 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.525060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.525509 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.525510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.525645 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.526253 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:51.526262 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:51.527816 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:51.530031 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:51.530041 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:51.530045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:51.530304 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:51.530572 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:51.530573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:51.530573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:51.530700 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:51.531017 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:51.531035 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:51.531195 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.531321 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.531506 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":132,"threads_started":1}
I20260812 00:06:51.532546 16594 heartbeater.cc:347] Connected to a master server at 127.7.85.254:40803
I20260812 00:06:51.532671 16594 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:51.532915 16594 heartbeater.cc:511] Master 127.7.85.254:40803 requested a full tablet report, sending...
I20260812 00:06:51.533432 15712 ts_manager.cc:194] Registered new tserver with Master: 446b7cc413c14a7b86aa84cddf4856b0 (127.7.85.195:39895)
I20260812 00:06:51.533783 15712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:54829
I20260812 00:06:51.545094 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:06:51.553820 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.546598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.7.85.254:40803)
0812 00:06:51.546677 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.546680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.546789 (+ 109us) server_negotiation.cc:407] Connection header received
0812 00:06:51.546835 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.546836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.546847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.546902 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:51.547460 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.547986 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.548745 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.548949 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.549010 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.550457 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:51.550459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:51.550459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:51.550466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:51.552354 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.552834 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.552835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.552836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.552906 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:51.553113 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:51.553114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:51.553114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:51.553376 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:51.553473 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.553666 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.553747 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:06:51.555889 15712 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:51.557648 15712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:51.564675 16608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.560428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:37701 (local address 127.0.0.1:48378)
0812 00:06:51.560626 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.560700 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.560773 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.561193 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.561197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.561199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:51.561275 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.561277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.562318 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.562319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.563483 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.563485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.564335 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.564430 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.564445 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.564539 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.564595 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":67,"threads_started":1}
I20260812 00:06:51.564687 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.560571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.7.85.193:37701)
0812 00:06:51.560989 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.560994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.561003 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:51.561038 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.561040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.561052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.561137 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:51.561340 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.562213 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.563580 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.564247 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.564398 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.564556 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.564596 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":86,"threads_started":1}
I20260812 00:06:51.565088 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.560483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:39895 (local address 127.0.0.1:33742)
0812 00:06:51.560826 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.560838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.560897 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.561495 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.561496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.561498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:51.561532 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.561533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.562839 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.562840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.563758 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.563760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.564838 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.564908 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.564912 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.564962 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.565003 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":25,"thread_start_us":101,"threads_started":1}
I20260812 00:06:51.565373 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.560606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.7.85.195:39895)
0812 00:06:51.561269 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.561273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.561283 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:06:51.561321 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.561323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.561338 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.561411 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:51.561612 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.562729 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.563846 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.564648 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564726 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.564775 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.565244 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.565295 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":101,"threads_started":1}
I20260812 00:06:51.565384 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.560689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:37329 (local address 127.0.0.1:45172)
0812 00:06:51.560901 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.560912 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.560953 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.561397 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.561398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.561400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:51.561459 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.561461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.562831 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.562832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.563937 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.563939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.564952 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.565013 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.565017 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.565156 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.565310 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":33,"thread_start_us":100,"threads_started":1}
I20260812 00:06:51.566139 16468 tablet_service.cc:1467] Processing CreateTablet for tablet 65a7dabf2a164a9191d945e96ec7c8d3 (DEFAULT_TABLE table=test-table [id=a2e3f6251c844e959491ad7917836161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:51.566156 15949 tablet_service.cc:1467] Processing CreateTablet for tablet 65a7dabf2a164a9191d945e96ec7c8d3 (DEFAULT_TABLE table=test-table [id=a2e3f6251c844e959491ad7917836161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:51.566514 16468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a7dabf2a164a9191d945e96ec7c8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:51.566512 15949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a7dabf2a164a9191d945e96ec7c8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:51.568722 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.560983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.7.85.194:37329)
0812 00:06:51.561194 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.561210 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.561223 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:06:51.561266 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.561268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.561285 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.561540 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:51.561618 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.562711 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564023 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.564867 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.564940 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.564996 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.565282 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.565334 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260812 00:06:51.569386 16625 tablet_bootstrap.cc:492] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: Bootstrap starting.
I20260812 00:06:51.569545 16206 tablet_service.cc:1467] Processing CreateTablet for tablet 65a7dabf2a164a9191d945e96ec7c8d3 (DEFAULT_TABLE table=test-table [id=a2e3f6251c844e959491ad7917836161]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:51.569792 16626 tablet_bootstrap.cc:492] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Bootstrap starting.
I20260812 00:06:51.570073 16206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65a7dabf2a164a9191d945e96ec7c8d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:51.570187 16625 tablet_bootstrap.cc:654] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:51.570470 16626 tablet_bootstrap.cc:654] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:51.570775 16625 log.cc:824] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:51.571082 16626 log.cc:824] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:51.571587 16625 tablet_bootstrap.cc:492] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: No bootstrap required, opened a new log
I20260812 00:06:51.571676 16625 ts_tablet_manager.cc:1397] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:51.571873 16626 tablet_bootstrap.cc:492] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: No bootstrap required, opened a new log
I20260812 00:06:51.571954 16626 ts_tablet_manager.cc:1397] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:51.573560 16625 raft_consensus.cc:348] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.573745 16625 raft_consensus.cc:374] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:51.573802 16625 raft_consensus.cc:729] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9fe443214da4fc68fd418789c111343, State: Initialized, Role: FOLLOWER
I20260812 00:06:51.573717 16630 tablet_bootstrap.cc:492] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Bootstrap starting.
I20260812 00:06:51.573948 16625 consensus_queue.cc:260] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.574095 16626 raft_consensus.cc:348] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.574255 16626 raft_consensus.cc:374] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:51.574263 16625 ts_tablet_manager.cc:1428] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:06:51.574302 16626 raft_consensus.cc:729] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 446b7cc413c14a7b86aa84cddf4856b0, State: Initialized, Role: FOLLOWER
I20260812 00:06:51.574419 16078 heartbeater.cc:503] Master 127.7.85.254:40803 was elected leader, sending a full tablet report...
I20260812 00:06:51.574455 16626 consensus_queue.cc:260] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.574635 16630 tablet_bootstrap.cc:654] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:51.574743 16626 ts_tablet_manager.cc:1428] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:06:51.574785 16594 heartbeater.cc:503] Master 127.7.85.254:40803 was elected leader, sending a full tablet report...
I20260812 00:06:51.575380 16630 log.cc:824] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:51.576205 16630 tablet_bootstrap.cc:492] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: No bootstrap required, opened a new log
I20260812 00:06:51.576295 16630 ts_tablet_manager.cc:1397] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:06:51.577811 16630 raft_consensus.cc:348] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.578014 16630 raft_consensus.cc:374] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:51.578068 16630 raft_consensus.cc:729] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3321e03dd6345f3b9ea1e81af383a4c, State: Initialized, Role: FOLLOWER
I20260812 00:06:51.578208 16630 consensus_queue.cc:260] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.578486 16630 ts_tablet_manager.cc:1428] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 00:06:51.578533 16346 heartbeater.cc:503] Master 127.7.85.254:40803 was elected leader, sending a full tablet report...
W20260812 00:06:51.631430 16080 tablet.cc:2369] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:51.734423 16632 raft_consensus.cc:482] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:51.734597 16632 raft_consensus.cc:504] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.735095 16632 leader_election.cc:288] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9fe443214da4fc68fd418789c111343 (127.7.85.193:37701), e3321e03dd6345f3b9ea1e81af383a4c (127.7.85.194:37329)
I20260812 00:06:51.740319 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.735690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:55125 (local address 127.7.85.193:37701)
0812 00:06:51.735762 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.735764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.735795 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:06:51.735829 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.735830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.735844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.735917 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:51.736323 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.737333 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.739090 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.739880 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.739977 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.739989 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.740182 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.740243 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:06:51.740332 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.735656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:54955 (local address 127.7.85.194:37329)
0812 00:06:51.735728 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.735731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.735803 (+ 72us) server_negotiation.cc:407] Connection header received
0812 00:06:51.735832 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.735833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.735846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.735917 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:51.736323 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.737333 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.739090 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.739880 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.739977 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.739989 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.740181 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.740243 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:06:51.740355 16597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.735375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:37701 (local address 127.7.85.195:55125)
0812 00:06:51.735447 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.735460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.735510 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.735989 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.735990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.735993 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:51.736103 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.736105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.737475 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.737477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.738932 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.738936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.740001 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.740002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.740088 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.740090 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.740156 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.740231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":123}
I20260812 00:06:51.740355 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.735389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:37329 (local address 127.7.85.195:54955)
0812 00:06:51.735696 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.735709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:51.735793 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:51.735989 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:51.735990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:51.735993 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:51.736108 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.736109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.737475 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.737477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.738935 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:51.738936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.740001 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.740002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:51.740085 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.740090 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:51.740156 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 00:06:51.740231 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":120,"thread_start_us":134,"threads_started":1}
I20260812 00:06:51.740780 15982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a7dabf2a164a9191d945e96ec7c8d3" candidate_uuid: "446b7cc413c14a7b86aa84cddf4856b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9fe443214da4fc68fd418789c111343" is_pre_election: true
I20260812 00:06:51.740810 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a7dabf2a164a9191d945e96ec7c8d3" candidate_uuid: "446b7cc413c14a7b86aa84cddf4856b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" is_pre_election: true
I20260812 00:06:51.740962 15982 raft_consensus.cc:2454] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 446b7cc413c14a7b86aa84cddf4856b0 in term 0.
I20260812 00:06:51.740988 16244 raft_consensus.cc:2454] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 446b7cc413c14a7b86aa84cddf4856b0 in term 0.
I20260812 00:06:51.741253 16393 leader_election.cc:302] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 446b7cc413c14a7b86aa84cddf4856b0, f9fe443214da4fc68fd418789c111343; no voters:
I20260812 00:06:51.741432 16632 raft_consensus.cc:2781] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:51.741494 16632 raft_consensus.cc:482] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:51.741533 16632 raft_consensus.cc:3037] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:51.742285 16632 raft_consensus.cc:504] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.742443 16632 leader_election.cc:288] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [CANDIDATE]: Term 1 election: Requested vote from peers f9fe443214da4fc68fd418789c111343 (127.7.85.193:37701), e3321e03dd6345f3b9ea1e81af383a4c (127.7.85.194:37329)
I20260812 00:06:51.742586 15982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a7dabf2a164a9191d945e96ec7c8d3" candidate_uuid: "446b7cc413c14a7b86aa84cddf4856b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9fe443214da4fc68fd418789c111343"
I20260812 00:06:51.742681 15982 raft_consensus.cc:3037] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:51.742875 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65a7dabf2a164a9191d945e96ec7c8d3" candidate_uuid: "446b7cc413c14a7b86aa84cddf4856b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3321e03dd6345f3b9ea1e81af383a4c"
I20260812 00:06:51.742978 16244 raft_consensus.cc:3037] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:51.743429 15982 raft_consensus.cc:2454] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 446b7cc413c14a7b86aa84cddf4856b0 in term 1.
I20260812 00:06:51.743618 16393 leader_election.cc:302] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 446b7cc413c14a7b86aa84cddf4856b0, f9fe443214da4fc68fd418789c111343; no voters:
I20260812 00:06:51.743728 16632 raft_consensus.cc:2781] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:51.743906 16632 raft_consensus.cc:686] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 1 LEADER]: Becoming Leader. State: Replica: 446b7cc413c14a7b86aa84cddf4856b0, State: Running, Role: LEADER
I20260812 00:06:51.744020 16632 consensus_queue.cc:237] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } }
I20260812 00:06:51.744266 16244 raft_consensus.cc:2454] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 446b7cc413c14a7b86aa84cddf4856b0 in term 1.
I20260812 00:06:51.744928 15710 catalog_manager.cc:5495] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 446b7cc413c14a7b86aa84cddf4856b0 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "446b7cc413c14a7b86aa84cddf4856b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "446b7cc413c14a7b86aa84cddf4856b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 } health_report { overall_health: UNKNOWN } } }
W20260812 00:06:51.771127 16596 tablet.cc:2369] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:51.796572 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.794082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33758 (local address 127.7.85.195:39895)
0812 00:06:51.794160 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:51.794161 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:51.794170 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:51.794218 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:51.794219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:51.794245 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:51.794336 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:06:51.794489 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.794968 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.795731 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:51.795927 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:51.796006 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:51.796185 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:06:51.796292 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:06:51.796388 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:51.796433 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 00:06:51.796455 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:06:51.798627 16244 raft_consensus.cc:1261] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 1 FOLLOWER]: Refusing update from remote peer 446b7cc413c14a7b86aa84cddf4856b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:51.798664 15982 raft_consensus.cc:1261] T 65a7dabf2a164a9191d945e96ec7c8d3 P f9fe443214da4fc68fd418789c111343 [term 1 FOLLOWER]: Refusing update from remote peer 446b7cc413c14a7b86aa84cddf4856b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:51.799046 16632 consensus_queue.cc:1035] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9fe443214da4fc68fd418789c111343" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:51.799188 16636 consensus_queue.cc:1035] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3321e03dd6345f3b9ea1e81af383a4c" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 37329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:51.802366 16645 mvcc.cc:205] Tried to move back new op lower bound from 7317476195523846144 to 7317476195304165376. Current Snapshot: MvccSnapshot[applied={T|T < 7317476195523846144}]
I20260812 00:06:51.802779 16643 mvcc.cc:205] Tried to move back new op lower bound from 7317476195523846144 to 7317476195304165376. Current Snapshot: MvccSnapshot[applied={T|T < 7317476195523846144 or (T in {7317476195523846144})}]
I20260812 00:06:51.804127 15712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:51.804248 15712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:51.805045 15712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271cd480983942a7b440c3171f532ee3: Sending DeleteTablet for 3 replicas of tablet 65a7dabf2a164a9191d945e96ec7c8d3
I20260812 00:06:51.805405 16468 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a7dabf2a164a9191d945e96ec7c8d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:51 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:33742
I20260812 00:06:51.805500 16206 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a7dabf2a164a9191d945e96ec7c8d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:51 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45172
I20260812 00:06:51.805546 15949 tablet_service.cc:1514] Processing DeleteTablet for tablet 65a7dabf2a164a9191d945e96ec7c8d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:51 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:48378
I20260812 00:06:51.805680 16660 tablet_replica.cc:330] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: stopping tablet replica
I20260812 00:06:51.805758 16661 tablet_replica.cc:330] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: stopping tablet replica
I20260812 00:06:51.805800 16660 raft_consensus.cc:2229] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:51.805852 16661 raft_consensus.cc:2229] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:51.805944 16660 raft_consensus.cc:2258] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:51.805999 16661 raft_consensus.cc:2258] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:51.806298 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15834
I20260812 00:06:51.806425 16660 ts_tablet_manager.cc:1905] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:51.806460 16661 ts_tablet_manager.cc:1905] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:51.808349 16661 ts_tablet_manager.cc:1918] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:51.808419 16661 log.cc:1196] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/65a7dabf2a164a9191d945e96ec7c8d3
I20260812 00:06:51.808705 16661 ts_tablet_manager.cc:1939] T 65a7dabf2a164a9191d945e96ec7c8d3 P e3321e03dd6345f3b9ea1e81af383a4c: Deleting consensus metadata
I20260812 00:06:51.809130 15688 catalog_manager.cc:4841] TS e3321e03dd6345f3b9ea1e81af383a4c (127.7.85.194:37329): tablet 65a7dabf2a164a9191d945e96ec7c8d3 (table test-table [id=a2e3f6251c844e959491ad7917836161]) successfully deleted
I20260812 00:06:51.811734 16660 ts_tablet_manager.cc:1918] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:51.811833 16660 log.cc:1196] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/65a7dabf2a164a9191d945e96ec7c8d3
I20260812 00:06:51.812162 16660 ts_tablet_manager.cc:1939] T 65a7dabf2a164a9191d945e96ec7c8d3 P 446b7cc413c14a7b86aa84cddf4856b0: Deleting consensus metadata
I20260812 00:06:51.812736 15688 catalog_manager.cc:4841] TS 446b7cc413c14a7b86aa84cddf4856b0 (127.7.85.195:39895): tablet 65a7dabf2a164a9191d945e96ec7c8d3 (table test-table [id=a2e3f6251c844e959491ad7917836161]) successfully deleted
W20260812 00:06:51.825956 15689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:37701 (error 108)
I20260812 00:06:51.826138 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16088
I20260812 00:06:51.829759 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:51.829589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:37701 (local address 127.0.0.1:48394)
0812 00:06:51.829662 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:51.829710 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 00:06:51.829905 15689 catalog_manager.cc:4584] TS f9fe443214da4fc68fd418789c111343 (127.7.85.193:37701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65a7dabf2a164a9191d945e96ec7c8d3: Network error: Client connection negotiation failed: client connection to 127.7.85.193:37701: connect: Connection refused (error 111)
I20260812 00:06:51.838033 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16356
I20260812 00:06:51.856047 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15660
2026-08-12T00:06:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2598 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T00:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:51Z Disabled control of system clock
I20260812 00:06:51.899032 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41733
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 00:06:51.985239 16667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:51.985531 16667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:51.985574 16667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:51.986785 16667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:51.986850 16667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:51.986874 16667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:51.986905 16667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:51.986930 16667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:51.988335 16667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41733
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:51.988631 16667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:51.988947 16667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:51.994131 16683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.994107 16679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:51.994115 16680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:51.994524 16667 server_base.cc:1034] running on GCE node
I20260812 00:06:51.994786 16667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:51.995041 16667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:51.996194 16667 hybrid_clock.cc:648] HybridClock initialized: now 1786493211996178 us; error 31 us; skew 500 ppm
I20260812 00:06:51.997614 16667 webserver.cc:460] Webserver started at http://127.7.85.254:42863/ using document root <none> and password file <none>
I20260812 00:06:51.997910 16667 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:51.997963 16667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:51.998106 16667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:51.999194 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "339aec786c1441c49ed4a7a27acc8649"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "d2270ca614e2ee2b9a28207b2d36a2fe"
server_key_iv: "1df4da0a63710501bcfb26d3f0994473"
server_key_version: "encryptionkey@0"
I20260812 00:06:51.999567 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "339aec786c1441c49ed4a7a27acc8649"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "d2270ca614e2ee2b9a28207b2d36a2fe"
server_key_iv: "1df4da0a63710501bcfb26d3f0994473"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.000974 16667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:52.002061 16694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.002300 16667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:52.002408 16667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "339aec786c1441c49ed4a7a27acc8649"
format_stamp: "Formatted at 2026-08-12 00:06:51 on dist-test-slave-z8x9"
server_key: "d2270ca614e2ee2b9a28207b2d36a2fe"
server_key_iv: "1df4da0a63710501bcfb26d3f0994473"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.002468 16667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:52.005966 16667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:52.006225 16667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:52.006330 16667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:52.010295 16667 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:41733
I20260812 00:06:52.010363 16780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:41733 every 8 connection(s)
I20260812 00:06:52.010725 16667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:52.011519 16781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:52.013999 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16667
I20260812 00:06:52.014129 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:52.014354 7511 external_mini_cluster.cc:1428] Setting key f80d268c3ec8c401b0020a51071c88d4
I20260812 00:06:52.014688 16781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649: Bootstrap starting.
I20260812 00:06:52.015702 16781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:52.016079 16781 log.cc:824] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:52.016963 16781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649: No bootstrap required, opened a new log
I20260812 00:06:52.017292 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.014914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.7.85.254:41733)
0812 00:06:52.015172 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.015180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.015208 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:06:52.015631 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.015637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.015675 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.015871 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.016660 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.016664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.016664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.016669 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.016718 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.016728 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.016864 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.016935 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.017003 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":89,"threads_started":1}
I20260812 00:06:52.018667 16781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339aec786c1441c49ed4a7a27acc8649" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41733 } }
I20260812 00:06:52.018826 16781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:52.018874 16781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339aec786c1441c49ed4a7a27acc8649, State: Initialized, Role: FOLLOWER
I20260812 00:06:52.018997 16781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339aec786c1441c49ed4a7a27acc8649" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41733 } }
I20260812 00:06:52.019446 16781 raft_consensus.cc:388] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:52.019500 16781 raft_consensus.cc:482] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:52.019554 16781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:52.020098 16781 raft_consensus.cc:504] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339aec786c1441c49ed4a7a27acc8649" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41733 } }
I20260812 00:06:52.020226 16781 leader_election.cc:302] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 339aec786c1441c49ed4a7a27acc8649; no voters:
I20260812 00:06:52.020418 16781 leader_election.cc:288] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:52.020596 16788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:52.020695 16781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:52.020774 16788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [term 1 LEADER]: Becoming Leader. State: Replica: 339aec786c1441c49ed4a7a27acc8649, State: Running, Role: LEADER
I20260812 00:06:52.020898 16788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339aec786c1441c49ed4a7a27acc8649" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41733 } }
I20260812 00:06:52.021308 16794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 339aec786c1441c49ed4a7a27acc8649. Latest consensus state: current_term: 1 leader_uuid: "339aec786c1441c49ed4a7a27acc8649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339aec786c1441c49ed4a7a27acc8649" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41733 } } }
I20260812 00:06:52.021409 16794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:52.021677 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "339aec786c1441c49ed4a7a27acc8649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339aec786c1441c49ed4a7a27acc8649" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41733 } } }
I20260812 00:06:52.021744 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:52.022171 16809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:52.022941 16809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:52.024411 16809 catalog_manager.cc:1347] Generated new cluster ID: a1966d995c5546baaaf842663c11fec8
I20260812 00:06:52.024474 16809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:52.041201 16809 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:52.041416 16809 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:52.043613 16809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649: Generated new TSK 0
I20260812 00:06:52.043802 16809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:06:52.051779 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41733
--builtin_ntp_servers=127.7.85.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 00:06:52.140476 16824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:52.140703 16824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:52.140726 16824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:52.142038 16824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:52.142088 16824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:52.142117 16824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:52.143546 16824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:52.143844 16824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:52.144178 16824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:52.150002 16848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.150024 16839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.150024 16840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:52.150596 16824 server_base.cc:1034] running on GCE node
I20260812 00:06:52.150869 16824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:52.151173 16824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:52.152436 16824 hybrid_clock.cc:648] HybridClock initialized: now 1786493212152418 us; error 36 us; skew 500 ppm
I20260812 00:06:52.154333 16824 webserver.cc:460] Webserver started at http://127.7.85.193:37097/ using document root <none> and password file <none>
I20260812 00:06:52.155247 16824 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:52.155383 16824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:52.155540 16824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:52.157717 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "414736e9507c4880b454010e81ace720"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "ceed50dcac5d6214392abc0b0d65b1d6"
server_key_iv: "35a803dcc0427e9b2c9117605c75dc12"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.158267 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "414736e9507c4880b454010e81ace720"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "ceed50dcac5d6214392abc0b0d65b1d6"
server_key_iv: "35a803dcc0427e9b2c9117605c75dc12"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.161338 16824 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:06:52.162635 16856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.162921 16824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:52.163051 16824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "414736e9507c4880b454010e81ace720"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "ceed50dcac5d6214392abc0b0d65b1d6"
server_key_iv: "35a803dcc0427e9b2c9117605c75dc12"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.163141 16824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:52.167062 16824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:52.167364 16824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:52.167512 16824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:52.167729 16824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:52.168057 16824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:52.168112 16824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.168154 16824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:52.168180 16824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.174430 16824 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:40197
I20260812 00:06:52.174510 17066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:40197 every 8 connection(s)
I20260812 00:06:52.174867 16824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:52.178049 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16824
I20260812 00:06:52.178176 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:52.178339 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.175730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:39097 (local address 127.7.85.254:41733)
0812 00:06:52.175816 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.175818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.176022 (+ 204us) server_negotiation.cc:407] Connection header received
0812 00:06:52.176501 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.176502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.176509 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.176578 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.177972 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.177974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.177974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.177980 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.177999 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.178004 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.178088 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.178255 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.178275 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:06:52.178380 7511 external_mini_cluster.cc:1428] Setting key e4c77af68677483e13009621274f9bfc
I20260812 00:06:52.178610 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.175675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41733 (local address 127.7.85.193:39097)
0812 00:06:52.175865 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.175907 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.176396 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.176646 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.176650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.176659 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.177737 (+ 1078us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.177738 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.177767 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.177796 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.178083 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.178090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.178094 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.178221 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.178298 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260812 00:06:52.179289 17069 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41733
I20260812 00:06:52.179430 17069 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:52.179566 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41733
--builtin_ntp_servers=127.7.85.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 00:06:52.179591 17069 heartbeater.cc:511] Master 127.7.85.254:41733 requested a full tablet report, sending...
I20260812 00:06:52.180128 16717 ts_manager.cc:194] Registered new tserver with Master: 414736e9507c4880b454010e81ace720 (127.7.85.193:40197)
W20260812 00:06:52.267993 17074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:52.268211 17074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:52.268234 17074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:52.269474 17074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:52.269519 17074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:52.269546 17074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:52.270910 17074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:52.271183 17074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:52.271497 17074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:52.277143 17083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.277122 17087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:52.277498 17074 server_base.cc:1034] running on GCE node
W20260812 00:06:52.277621 17092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:52.277922 17074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:52.278227 17074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:52.279414 17074 hybrid_clock.cc:648] HybridClock initialized: now 1786493212279382 us; error 51 us; skew 500 ppm
I20260812 00:06:52.281337 17074 webserver.cc:460] Webserver started at http://127.7.85.194:41787/ using document root <none> and password file <none>
I20260812 00:06:52.281884 17074 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:52.281960 17074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:52.282114 17074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:52.283283 17074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "ad2b4d31aa5a421c839b61372a7a0812"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "fdba9e1556e376e2e665f65a2d09a0c3"
server_key_iv: "4b750eab4ce96914ca6d8300056e64ed"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.283665 17074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "ad2b4d31aa5a421c839b61372a7a0812"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "fdba9e1556e376e2e665f65a2d09a0c3"
server_key_iv: "4b750eab4ce96914ca6d8300056e64ed"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.285392 17074 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 00:06:52.286360 17106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.286624 17074 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:52.286720 17074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "ad2b4d31aa5a421c839b61372a7a0812"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "fdba9e1556e376e2e665f65a2d09a0c3"
server_key_iv: "4b750eab4ce96914ca6d8300056e64ed"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.286826 17074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:52.290490 17074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:52.290777 17074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:52.290931 17074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:52.291150 17074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:52.291491 17074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:52.291549 17074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.291591 17074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:52.291618 17074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.298141 17074 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:45185
I20260812 00:06:52.298234 17329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:45185 every 8 connection(s)
I20260812 00:06:52.298599 17074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:06:52.302172 17334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.299461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41733 (local address 127.7.85.194:56871)
0812 00:06:52.299651 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.299712 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.300177 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.300466 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.300469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.300478 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.301566 (+ 1088us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.301567 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.301597 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.301626 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.301875 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.301880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.301882 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.301944 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.301995 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260812 00:06:52.302192 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.299465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:56871 (local address 127.7.85.254:41733)
0812 00:06:52.299537 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.299539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.299891 (+ 352us) server_negotiation.cc:407] Connection header received
0812 00:06:52.300315 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.300316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.300323 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.300391 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.301771 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.301773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.301773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.301778 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.301793 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.301797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.301866 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.302058 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.302086 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:06:52.302942 17331 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41733
I20260812 00:06:52.303036 17331 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:52.303193 17331 heartbeater.cc:511] Master 127.7.85.254:41733 requested a full tablet report, sending...
I20260812 00:06:52.303642 16717 ts_manager.cc:194] Registered new tserver with Master: ad2b4d31aa5a421c839b61372a7a0812 (127.7.85.194:45185)
I20260812 00:06:52.305966 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17074
I20260812 00:06:52.306093 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:52.306269 7511 external_mini_cluster.cc:1428] Setting key d790b43f7cc95cc8cc4fdc7007238ae9
I20260812 00:06:52.307436 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41733
--builtin_ntp_servers=127.7.85.212:37373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 00:06:52.400650 17335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:52.401008 17335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:52.401038 17335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:52.404089 17335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:52.404237 17335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:52.404330 17335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:52.406450 17335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:52.406703 17335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:52.407065 17335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:52.412642 17349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.412676 17345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.412642 17346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:52.413061 17335 server_base.cc:1034] running on GCE node
I20260812 00:06:52.413416 17335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:52.413690 17335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:52.415222 17335 hybrid_clock.cc:648] HybridClock initialized: now 1786493212415208 us; error 38 us; skew 500 ppm
I20260812 00:06:52.417099 17335 webserver.cc:460] Webserver started at http://127.7.85.195:42403/ using document root <none> and password file <none>
I20260812 00:06:52.417431 17335 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:52.417514 17335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:52.417693 17335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:52.418848 17335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "9c461f56629a43069170905e8b4c4bbd"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "9b2eff9f5a909163504153ce7d2d2b30"
server_key_iv: "b87472b3c9425ab9be9945e5acaa8994"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.419229 17335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "9c461f56629a43069170905e8b4c4bbd"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "9b2eff9f5a909163504153ce7d2d2b30"
server_key_iv: "b87472b3c9425ab9be9945e5acaa8994"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.420886 17335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:52.422830 17362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.423151 17335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:52.423255 17335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "9c461f56629a43069170905e8b4c4bbd"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "9b2eff9f5a909163504153ce7d2d2b30"
server_key_iv: "b87472b3c9425ab9be9945e5acaa8994"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.423381 17335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:52.427165 17335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:52.427466 17335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:52.427639 17335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:52.427893 17335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:52.428296 17335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:52.428355 17335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.428385 17335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:52.428406 17335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.435599 17335 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:33199
I20260812 00:06:52.435904 17583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:33199 every 8 connection(s)
I20260812 00:06:52.436055 17335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:52.440785 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.437227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:43913 (local address 127.7.85.254:41733)
0812 00:06:52.437320 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.437323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.437795 (+ 472us) server_negotiation.cc:407] Connection header received
0812 00:06:52.438295 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.438296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.438305 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.438376 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.440290 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.440291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.440292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.440297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.440313 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.440318 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.440383 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.440686 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.440707 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:06:52.441279 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.436998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41733 (local address 127.7.85.195:43913)
0812 00:06:52.437481 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.437634 (+ 153us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.438175 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.438454 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.438457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.438467 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.440048 (+ 1581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.440049 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.440095 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.440129 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.440440 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.440448 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.440453 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.440689 (+ 236us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.440795 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":243,"threads_started":1}
I20260812 00:06:52.441905 17584 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41733
I20260812 00:06:52.442013 17584 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:52.442150 17584 heartbeater.cc:511] Master 127.7.85.254:41733 requested a full tablet report, sending...
I20260812 00:06:52.442721 16717 ts_manager.cc:194] Registered new tserver with Master: 9c461f56629a43069170905e8b4c4bbd (127.7.85.195:33199)
I20260812 00:06:52.445578 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17335
I20260812 00:06:52.445679 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:52.445818 7511 external_mini_cluster.cc:1428] Setting key b104d5b570babb497a6b79e45707011a
I20260812 00:06:52.447430 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:06:52.450816 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.449371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.7.85.254:41733)
0812 00:06:52.449435 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.449437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.449688 (+ 251us) server_negotiation.cc:407] Connection header received
0812 00:06:52.449789 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.449789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.449796 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.449856 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.450513 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.450514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.450515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.450520 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.450537 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.450547 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.450606 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.450689 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.450706 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 00:06:52.453500 16717 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:06:52.455142 16717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:06:52.460741 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.458581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.7.85.193:40197)
0812 00:06:52.458801 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.458806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.458819 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:06:52.458849 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.458851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.458862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.458935 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.460318 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.460327 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.460327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.460332 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.460364 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.460386 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.460458 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.460629 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.460646 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
I20260812 00:06:52.460752 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.458206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40197 (local address 127.0.0.1:58068)
0812 00:06:52.458534 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.458554 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.458618 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.458935 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.458941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.458948 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.460167 (+ 1219us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.460168 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.460172 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.460192 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.460521 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.460523 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.460524 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.460599 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.460620 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260812 00:06:52.460752 17608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.458233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:33199 (local address 127.0.0.1:41698)
0812 00:06:52.458448 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.458495 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.458554 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.459638 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.459641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.459647 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.460278 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.460279 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.460290 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.460308 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.460650 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.460650 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.460651 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.460680 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.460691 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
I20260812 00:06:52.460896 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.458429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:45185 (local address 127.0.0.1:55242)
0812 00:06:52.458804 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.458815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.458859 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.459583 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.459584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.459587 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.460143 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.460144 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.460164 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.460193 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.460468 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.460474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.460478 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.460535 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.460549 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":64,"threads_started":1}
I20260812 00:06:52.461045 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.458705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.7.85.195:33199)
0812 00:06:52.459294 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.459301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.459316 (+ 15us) server_negotiation.cc:407] Connection header received
0812 00:06:52.459356 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.459358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.459373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.459906 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.460495 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.460501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.460501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.460505 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.460548 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.460569 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.460795 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.460905 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.460925 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":151,"threads_started":1}
I20260812 00:06:52.461853 16936 tablet_service.cc:1467] Processing CreateTablet for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 (DEFAULT_TABLE table=test-table [id=9c1a90e76c8842e2a93e05e526becbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:52.462088 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.458814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.7.85.194:45185)
0812 00:06:52.459416 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.459421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.459432 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:52.459460 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.459462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.459472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.459569 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.460300 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.460308 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.460308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.460314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.460358 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.460381 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.460686 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.460709 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.460725 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":84,"threads_started":1}
I20260812 00:06:52.462323 16936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:52.462366 17452 tablet_service.cc:1467] Processing CreateTablet for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 (DEFAULT_TABLE table=test-table [id=9c1a90e76c8842e2a93e05e526becbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:52.462770 17203 tablet_service.cc:1467] Processing CreateTablet for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 (DEFAULT_TABLE table=test-table [id=9c1a90e76c8842e2a93e05e526becbc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:06:52.463029 17452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:52.463143 17203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:52.466019 17615 tablet_bootstrap.cc:492] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Bootstrap starting.
I20260812 00:06:52.466898 17615 tablet_bootstrap.cc:654] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:52.467022 17618 tablet_bootstrap.cc:492] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Bootstrap starting.
I20260812 00:06:52.467257 17616 tablet_bootstrap.cc:492] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720: Bootstrap starting.
I20260812 00:06:52.467264 17615 log.cc:824] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:52.467738 17618 tablet_bootstrap.cc:654] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:52.467967 17616 tablet_bootstrap.cc:654] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:52.468050 17618 log.cc:824] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:52.468282 17615 tablet_bootstrap.cc:492] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: No bootstrap required, opened a new log
I20260812 00:06:52.468358 17615 ts_tablet_manager.cc:1397] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:52.468344 17616 log.cc:824] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:52.468842 17618 tablet_bootstrap.cc:492] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: No bootstrap required, opened a new log
I20260812 00:06:52.468978 17618 ts_tablet_manager.cc:1397] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:06:52.469034 17616 tablet_bootstrap.cc:492] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720: No bootstrap required, opened a new log
I20260812 00:06:52.469091 17616 ts_tablet_manager.cc:1397] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:06:52.471087 17616 raft_consensus.cc:348] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.471252 17616 raft_consensus.cc:374] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:52.471125 17615 raft_consensus.cc:348] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.471277 17616 raft_consensus.cc:729] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 414736e9507c4880b454010e81ace720, State: Initialized, Role: FOLLOWER
I20260812 00:06:52.471302 17615 raft_consensus.cc:374] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:52.471338 17615 raft_consensus.cc:729] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c461f56629a43069170905e8b4c4bbd, State: Initialized, Role: FOLLOWER
I20260812 00:06:52.471401 17616 consensus_queue.cc:260] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.471449 17615 consensus_queue.cc:260] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.471707 17616 ts_tablet_manager.cc:1428] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 00:06:52.471724 17069 heartbeater.cc:503] Master 127.7.85.254:41733 was elected leader, sending a full tablet report...
I20260812 00:06:52.471750 17615 ts_tablet_manager.cc:1428] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 00:06:52.471875 17584 heartbeater.cc:503] Master 127.7.85.254:41733 was elected leader, sending a full tablet report...
I20260812 00:06:52.472460 17618 raft_consensus.cc:348] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.472653 17618 raft_consensus.cc:374] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:52.472700 17618 raft_consensus.cc:729] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2b4d31aa5a421c839b61372a7a0812, State: Initialized, Role: FOLLOWER
I20260812 00:06:52.472818 17618 consensus_queue.cc:260] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.473278 17618 ts_tablet_manager.cc:1428] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 00:06:52.473344 17331 heartbeater.cc:503] Master 127.7.85.254:41733 was elected leader, sending a full tablet report...
W20260812 00:06:52.549350 17333 tablet.cc:2369] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:06:52.567209 17627 raft_consensus.cc:482] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:52.567370 17627 raft_consensus.cc:504] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.567901 17627 leader_election.cc:288] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 414736e9507c4880b454010e81ace720 (127.7.85.193:40197), ad2b4d31aa5a421c839b61372a7a0812 (127.7.85.194:45185)
I20260812 00:06:52.569422 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.568138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:45185 (local address 127.7.85.195:53871)
0812 00:06:52.568200 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.568215 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.568261 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.568500 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.568501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.568504 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.569043 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.569044 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.569053 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.569068 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.569307 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.569308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.569309 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.569354 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.569368 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:06:52.569576 17614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.568327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:53871 (local address 127.7.85.194:45185)
0812 00:06:52.568376 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.568383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.568391 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:06:52.568417 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.568418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.568422 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.568492 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.569161 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.569163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.569164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.569203 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.569223 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.569228 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.569296 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.569483 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.569499 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:06:52.569614 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.568233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:39059 (local address 127.7.85.193:40197)
0812 00:06:52.568313 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.568315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.568593 (+ 278us) server_negotiation.cc:407] Connection header received
0812 00:06:52.568682 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.568683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.568691 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.568789 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.569368 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.569369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.569370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.569374 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.569389 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.569393 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.569453 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.569529 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.569542 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:06:52.569841 17634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.568153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40197 (local address 127.7.85.195:39059)
0812 00:06:52.568508 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.568524 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.568602 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.568800 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.568801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.568803 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:52.569280 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:06:52.569281 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:52.569288 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:06:52.569302 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:52.569451 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:52.569452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:52.569452 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 00:06:52.569727 (+ 275us) client_negotiation.cc:240] Negotiation successful
0812 00:06:52.569741 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":181,"thread_start_us":114,"threads_started":1}
I20260812 00:06:52.569962 17240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f0d510a9e54eebb4830c4dc1a3ffe7" candidate_uuid: "9c461f56629a43069170905e8b4c4bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2b4d31aa5a421c839b61372a7a0812" is_pre_election: true
I20260812 00:06:52.570137 17240 raft_consensus.cc:2454] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c461f56629a43069170905e8b4c4bbd in term 0.
I20260812 00:06:52.570163 16981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f0d510a9e54eebb4830c4dc1a3ffe7" candidate_uuid: "9c461f56629a43069170905e8b4c4bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "414736e9507c4880b454010e81ace720" is_pre_election: true
I20260812 00:06:52.570340 16981 raft_consensus.cc:2454] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c461f56629a43069170905e8b4c4bbd in term 0.
I20260812 00:06:52.570420 17364 leader_election.cc:302] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c461f56629a43069170905e8b4c4bbd, ad2b4d31aa5a421c839b61372a7a0812; no voters:
I20260812 00:06:52.570578 17627 raft_consensus.cc:2781] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:06:52.570641 17627 raft_consensus.cc:482] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:06:52.570658 17627 raft_consensus.cc:3037] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:52.571455 17627 raft_consensus.cc:504] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.571688 17627 leader_election.cc:288] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [CANDIDATE]: Term 1 election: Requested vote from peers 414736e9507c4880b454010e81ace720 (127.7.85.193:40197), ad2b4d31aa5a421c839b61372a7a0812 (127.7.85.194:45185)
I20260812 00:06:52.571769 17240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f0d510a9e54eebb4830c4dc1a3ffe7" candidate_uuid: "9c461f56629a43069170905e8b4c4bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2b4d31aa5a421c839b61372a7a0812"
I20260812 00:06:52.571844 17240 raft_consensus.cc:3037] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:52.571879 16981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f0d510a9e54eebb4830c4dc1a3ffe7" candidate_uuid: "9c461f56629a43069170905e8b4c4bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "414736e9507c4880b454010e81ace720"
I20260812 00:06:52.571951 16981 raft_consensus.cc:3037] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:52.572512 17240 raft_consensus.cc:2454] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c461f56629a43069170905e8b4c4bbd in term 1.
I20260812 00:06:52.572753 16981 raft_consensus.cc:2454] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c461f56629a43069170905e8b4c4bbd in term 1.
I20260812 00:06:52.572795 17364 leader_election.cc:302] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c461f56629a43069170905e8b4c4bbd, ad2b4d31aa5a421c839b61372a7a0812; no voters:
I20260812 00:06:52.572947 17627 raft_consensus.cc:2781] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:52.573143 17627 raft_consensus.cc:686] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 1 LEADER]: Becoming Leader. State: Replica: 9c461f56629a43069170905e8b4c4bbd, State: Running, Role: LEADER
I20260812 00:06:52.573293 17627 consensus_queue.cc:237] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } }
I20260812 00:06:52.574373 16716 catalog_manager.cc:5495] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c461f56629a43069170905e8b4c4bbd (127.7.85.195). New cstate: current_term: 1 leader_uuid: "9c461f56629a43069170905e8b4c4bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c461f56629a43069170905e8b4c4bbd" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 33199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 } health_report { overall_health: UNKNOWN } } }
I20260812 00:06:52.582916 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.581667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.7.85.195:33199)
0812 00:06:52.581735 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.581737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.581746 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:52.581824 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.581825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.581839 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.581889 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.582578 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.582579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.582580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.582584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.582602 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.582606 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.582666 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.582785 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.582796 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:06:52.585047 17240 raft_consensus.cc:1261] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 1 FOLLOWER]: Refusing update from remote peer 9c461f56629a43069170905e8b4c4bbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:52.585047 16981 raft_consensus.cc:1261] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 414736e9507c4880b454010e81ace720 [term 1 FOLLOWER]: Refusing update from remote peer 9c461f56629a43069170905e8b4c4bbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:06:52.585493 17627 consensus_queue.cc:1035] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2b4d31aa5a421c839b61372a7a0812" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:52.585680 17635 consensus_queue.cc:1035] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "414736e9507c4880b454010e81ace720" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:52.589145 17646 mvcc.cc:205] Tried to move back new op lower bound from 7317476198744702976 to 7317476198701101056. Current Snapshot: MvccSnapshot[applied={T|T < 7317476198744702976 or (T in {7317476198744702976})}]
I20260812 00:06:52.589303 17641 mvcc.cc:205] Tried to move back new op lower bound from 7317476198744702976 to 7317476198701101056. Current Snapshot: MvccSnapshot[applied={T|T < 7317476198744702976}]
I20260812 00:06:52.591959 16716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:52.592070 16716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:06:52.593325 16716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 339aec786c1441c49ed4a7a27acc8649: Sending DeleteTablet for 3 replicas of tablet 49f0d510a9e54eebb4830c4dc1a3ffe7
I20260812 00:06:52.594022 16936 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:52 UTC) from {username='slave'} at 127.0.0.1:58068
I20260812 00:06:52.594058 17452 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:52 UTC) from {username='slave'} at 127.0.0.1:41698
I20260812 00:06:52.594388 17663 tablet_replica.cc:330] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: stopping tablet replica
I20260812 00:06:52.594419 17203 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:06:52 UTC) from {username='slave'} at 127.0.0.1:55242
I20260812 00:06:52.594489 17663 raft_consensus.cc:2229] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:06:52.594609 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16824
I20260812 00:06:52.594621 17663 raft_consensus.cc:2258] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:52.605522 17663 ts_tablet_manager.cc:1905] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:52.607630 17663 ts_tablet_manager.cc:1918] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:52.607733 17663 log.cc:1196] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/49f0d510a9e54eebb4830c4dc1a3ffe7
I20260812 00:06:52.608028 17663 ts_tablet_manager.cc:1939] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P 9c461f56629a43069170905e8b4c4bbd: Deleting consensus metadata
I20260812 00:06:52.608523 17668 tablet_replica.cc:330] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: stopping tablet replica
I20260812 00:06:52.608587 16695 catalog_manager.cc:4841] TS 9c461f56629a43069170905e8b4c4bbd (127.7.85.195:33199): tablet 49f0d510a9e54eebb4830c4dc1a3ffe7 (table test-table [id=9c1a90e76c8842e2a93e05e526becbc7]) successfully deleted
I20260812 00:06:52.608664 17668 raft_consensus.cc:2229] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:52.608784 17668 raft_consensus.cc:2258] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:52.609357 17668 ts_tablet_manager.cc:1905] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 00:06:52.610615 16700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:40197 (error 108)
I20260812 00:06:52.610941 17668 ts_tablet_manager.cc:1918] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:06:52.611017 17668 log.cc:1196] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/49f0d510a9e54eebb4830c4dc1a3ffe7
I20260812 00:06:52.611009 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17074
I20260812 00:06:52.611169 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.611032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40197 (local address 127.0.0.1:58084)
0812 00:06:52.611090 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.611129 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:40197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 00:06:52.611285 17668 ts_tablet_manager.cc:1939] T 49f0d510a9e54eebb4830c4dc1a3ffe7 P ad2b4d31aa5a421c839b61372a7a0812: Deleting consensus metadata
W20260812 00:06:52.611274 16700 catalog_manager.cc:4584] TS 414736e9507c4880b454010e81ace720 (127.7.85.193:40197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49f0d510a9e54eebb4830c4dc1a3ffe7: Network error: Client connection negotiation failed: client connection to 127.7.85.193:40197: connect: Connection refused (error 111)
I20260812 00:06:52.622507 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17335
I20260812 00:06:52.622555 17603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.622365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:45185 (local address 127.0.0.1:55254)
0812 00:06:52.622427 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.622476 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.194:45185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 00:06:52.637547 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16667
2026-08-12T00:06:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (781 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 00:06:52.670521 7511 mini_oidc.cc:152] Starting JWKS server
I20260812 00:06:52.672375 7511 webserver.cc:460] Webserver started at https://127.0.0.1:39609/ using document root <none> and password file <none>
I20260812 00:06:52.672498 7511 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 00:06:52.672700 7511 webserver.cc:460] Webserver started at http://127.0.0.1:37015/ using document root <none> and password file <none>
2026-08-12T00:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:52Z Disabled control of system clock
I20260812 00:06:52.685151 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36285
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36285
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39609/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/testchainca.cert with env {}
W20260812 00:06:52.779732 17680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:52.780084 17680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:52.780112 17680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:52.781451 17680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:52.781509 17680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:52.781522 17680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:52.781536 17680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:52.781548 17680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:06:52.781561 17680 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:06:52.781572 17680 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39609/jwks.json
I20260812 00:06:52.783931 17680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39609/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:52.784209 17680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:52.784395 17680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:52.790748 17688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.791254 17690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:52.791319 17687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:52.791728 17680 server_base.cc:1034] running on GCE node
I20260812 00:06:52.792387 17680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:52.792842 17680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:52.794199 17680 hybrid_clock.cc:648] HybridClock initialized: now 1786493212794133 us; error 69 us; skew 500 ppm
I20260812 00:06:52.796137 17680 webserver.cc:460] Webserver started at http://127.7.85.254:42965/ using document root <none> and password file <none>
I20260812 00:06:52.796500 17680 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:52.796581 17680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:52.796752 17680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:52.798290 17680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "b784628fb6da489b91cc40320fec1c7b"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "dd76a913f817c2903d7858c9fbe89d1c"
server_key_iv: "f8a78f86ecb8a198cc025e116533dedb"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.798743 17680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "b784628fb6da489b91cc40320fec1c7b"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "dd76a913f817c2903d7858c9fbe89d1c"
server_key_iv: "f8a78f86ecb8a198cc025e116533dedb"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.800305 17680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:52.801548 17703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:52.801839 17680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 00:06:52.801970 17680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "b784628fb6da489b91cc40320fec1c7b"
format_stamp: "Formatted at 2026-08-12 00:06:52 on dist-test-slave-z8x9"
server_key: "dd76a913f817c2903d7858c9fbe89d1c"
server_key_iv: "f8a78f86ecb8a198cc025e116533dedb"
server_key_version: "encryptionkey@0"
I20260812 00:06:52.802074 17680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:52.810022 17680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:52.810418 17680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:52.810595 17680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:52.816221 17680 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:36285
I20260812 00:06:52.816320 17786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:36285 every 8 connection(s)
I20260812 00:06:52.817060 17680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:52.818542 17788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:52.821398 17788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b: Bootstrap starting.
I20260812 00:06:52.822078 17788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:52.822414 17788 log.cc:824] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:52.823107 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17680
I20260812 00:06:52.823272 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:52.823674 17788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b: No bootstrap required, opened a new log
I20260812 00:06:52.823674 7511 external_mini_cluster.cc:1428] Setting key f75c8339d23de8ba175272e3d1c2b736
I20260812 00:06:52.827039 17788 raft_consensus.cc:348] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b784628fb6da489b91cc40320fec1c7b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36285 } }
I20260812 00:06:52.827243 17788 raft_consensus.cc:374] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:52.827611 17788 raft_consensus.cc:729] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b784628fb6da489b91cc40320fec1c7b, State: Initialized, Role: FOLLOWER
I20260812 00:06:52.827829 17788 consensus_queue.cc:260] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b784628fb6da489b91cc40320fec1c7b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36285 } }
I20260812 00:06:52.827946 17788 raft_consensus.cc:388] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:52.827987 17788 raft_consensus.cc:482] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:52.828033 17788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:52.829634 17788 raft_consensus.cc:504] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b784628fb6da489b91cc40320fec1c7b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36285 } }
I20260812 00:06:52.829818 17788 leader_election.cc:302] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b784628fb6da489b91cc40320fec1c7b; no voters:
I20260812 00:06:52.830017 17788 leader_election.cc:288] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:52.830272 17799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:52.830315 17788 sys_catalog.cc:564] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:52.830586 17799 raft_consensus.cc:686] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [term 1 LEADER]: Becoming Leader. State: Replica: b784628fb6da489b91cc40320fec1c7b, State: Running, Role: LEADER
I20260812 00:06:52.830734 17799 consensus_queue.cc:237] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b784628fb6da489b91cc40320fec1c7b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36285 } }
I20260812 00:06:52.831564 17803 sys_catalog.cc:455] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b784628fb6da489b91cc40320fec1c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b784628fb6da489b91cc40320fec1c7b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36285 } } }
I20260812 00:06:52.831676 17803 sys_catalog.cc:458] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:52.831993 17807 sys_catalog.cc:455] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b784628fb6da489b91cc40320fec1c7b. Latest consensus state: current_term: 1 leader_uuid: "b784628fb6da489b91cc40320fec1c7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b784628fb6da489b91cc40320fec1c7b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36285 } } }
I20260812 00:06:52.832113 17807 sys_catalog.cc:458] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b [sys.catalog]: This master's current role is: LEADER
W20260812 00:06:52.832911 17833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:06:52.833004 17833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:06:52.833084 17825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:52.833293 17825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:52.837713 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.824723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.7.85.254:36285)
0812 00:06:52.825026 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.825038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.825066 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:06:52.827028 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.827040 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.827132 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.827455 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:52.827911 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:52.830356 (+ 2445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:52.831793 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:52.832596 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:52.832845 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:52.834441 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:52.834448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:52.834448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:52.834837 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:52.835472 (+ 635us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:52.835966 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:52.836215 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:52.836640 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 00:06:52.837452 (+ 812us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":130,"threads_started":1}
I20260812 00:06:52.837862 17825 catalog_manager.cc:1347] Generated new cluster ID: 672b434d068e4dfe999b7ef09d56ce77
I20260812 00:06:52.837919 17825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:52.839461 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:06:52.854988 17825 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:52.855906 17825 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:52.864142 17825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b784628fb6da489b91cc40320fec1c7b: Generated new TSK 0
I20260812 00:06:52.864405 17825 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 00:06:52.896819 17853 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 00:06:52.897070 17853 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 00:06:52.897518 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:52.879082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.7.85.254:36285)
0812 00:06:52.879158 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:52.879161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:52.879408 (+ 247us) server_negotiation.cc:407] Connection header received
0812 00:06:52.879461 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:52.879462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:52.879481 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:52.879792 (+ 311us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:06:52.880681 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:52.881514 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:52.882811 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:52.883015 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:52.883084 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:52.883498 (+ 414us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:06:52.896925 (+ 13427us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 00:06:52.897417 (+ 492us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 00:06:52.897637 17796 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 00:06:52.898224 17852 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:52.879009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36285 (local address 127.0.0.1:35318)
0812 00:06:52.879248 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:52.879262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:52.879351 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:52.879815 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:52.879823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:52.879830 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:06:52.879907 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:52.879910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:52.881643 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:52.881644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:52.882673 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:52.882677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:52.883136 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:52.883138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:52.883217 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:52.883289 (+ 72us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:06:52.897455 (+ 14166us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 00:06:52.897613 (+ 158us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.85.254:36285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.85.254:36285: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39609/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 00:06:52.898928 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17680
2026-08-12T00:06:52Z chronyd exiting
I20260812 00:06:52.943586 7511 test_util.cc:175] -----------------------------------------------
I20260812 00:06:52.943694 7511 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786493185917310-7511-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (274 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 00:06:52.944670 7511 mini_oidc.cc:152] Starting JWKS server
I20260812 00:06:52.945501 7511 webserver.cc:460] Webserver started at https://127.0.0.1:35051/ using document root <none> and password file <none>
I20260812 00:06:52.945547 7511 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 00:06:52.945717 7511 webserver.cc:460] Webserver started at http://127.0.0.1:37587/ using document root <none> and password file <none>
2026-08-12T00:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:52Z Disabled control of system clock
I20260812 00:06:52.957657 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:46113
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:46113
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35051/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/testchainca.cert with env {}
W20260812 00:06:53.047358 17861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:53.047884 17861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:53.047966 17861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:53.049542 17861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:53.049603 17861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:53.049618 17861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:53.049723 17861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:53.049760 17861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:06:53.049777 17861 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:06:53.049789 17861 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35051/jwks.json
I20260812 00:06:53.051566 17861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:46113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:46113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35051/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:53.051817 17861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:53.051999 17861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:53.057380 17882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:53.057384 17879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:53.057780 17876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:53.058018 17861 server_base.cc:1034] running on GCE node
I20260812 00:06:53.058319 17861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:53.058657 17861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:53.059841 17861 hybrid_clock.cc:648] HybridClock initialized: now 1786493213059814 us; error 52 us; skew 500 ppm
I20260812 00:06:53.061579 17861 webserver.cc:460] Webserver started at http://127.7.85.254:40899/ using document root <none> and password file <none>
I20260812 00:06:53.061856 17861 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:53.061913 17861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:53.062062 17861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:53.063216 17861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "d9daf0d3a05d411fb54bef12facf43a4"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "9c3cf847ce8ed273442f392876e35ed2"
server_key_iv: "f428b1f4198adb814f63f64497c439cd"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.063599 17861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "d9daf0d3a05d411fb54bef12facf43a4"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "9c3cf847ce8ed273442f392876e35ed2"
server_key_iv: "f428b1f4198adb814f63f64497c439cd"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.065332 17861 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 00:06:53.066547 17893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:53.066917 17861 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:53.067219 17861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "d9daf0d3a05d411fb54bef12facf43a4"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "9c3cf847ce8ed273442f392876e35ed2"
server_key_iv: "f428b1f4198adb814f63f64497c439cd"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.067349 17861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:53.083501 17861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:53.083920 17861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:53.084144 17861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:53.088831 17861 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:46113
I20260812 00:06:53.088873 17983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:46113 every 8 connection(s)
I20260812 00:06:53.089303 17861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:53.090456 17984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:53.094069 17984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4: Bootstrap starting.
I20260812 00:06:53.094470 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17861
I20260812 00:06:53.094606 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:53.094815 7511 external_mini_cluster.cc:1428] Setting key b616d26de4a4f8596e0513025cc974f8
I20260812 00:06:53.094817 17984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:53.095136 17984 log.cc:824] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:53.096004 17984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4: No bootstrap required, opened a new log
I20260812 00:06:53.100167 17984 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf0d3a05d411fb54bef12facf43a4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46113 } }
I20260812 00:06:53.100381 17984 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:53.100422 17984 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9daf0d3a05d411fb54bef12facf43a4, State: Initialized, Role: FOLLOWER
I20260812 00:06:53.100612 17984 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf0d3a05d411fb54bef12facf43a4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46113 } }
I20260812 00:06:53.100730 17984 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:53.100772 17984 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:53.100833 17984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:53.102419 17984 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf0d3a05d411fb54bef12facf43a4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46113 } }
I20260812 00:06:53.102476 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:53.095496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.7.85.254:46113)
0812 00:06:53.095746 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:53.095754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:53.095783 (+ 29us) server_negotiation.cc:407] Connection header received
0812 00:06:53.096357 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:53.096365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:53.096429 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:53.096549 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:53.097102 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.098101 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.099794 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.100401 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.100617 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.101756 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:53.101760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:53.101760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:53.101947 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:53.101982 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:53.101995 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:53.102053 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:53.102168 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 00:06:53.102268 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260812 00:06:53.102576 17984 leader_election.cc:302] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9daf0d3a05d411fb54bef12facf43a4; no voters:
I20260812 00:06:53.102802 17984 leader_election.cc:288] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:53.102865 17990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:53.103024 17990 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [term 1 LEADER]: Becoming Leader. State: Replica: d9daf0d3a05d411fb54bef12facf43a4, State: Running, Role: LEADER
I20260812 00:06:53.103119 17984 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:53.103114 17990 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf0d3a05d411fb54bef12facf43a4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46113 } }
I20260812 00:06:53.103688 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:06:53.103790 17990 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9daf0d3a05d411fb54bef12facf43a4. Latest consensus state: current_term: 1 leader_uuid: "d9daf0d3a05d411fb54bef12facf43a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf0d3a05d411fb54bef12facf43a4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46113 } } }
I20260812 00:06:53.103892 17990 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:53.104182 18004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:53.104730 18004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:53.104935 17993 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9daf0d3a05d411fb54bef12facf43a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf0d3a05d411fb54bef12facf43a4" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46113 } } }
I20260812 00:06:53.105259 17993 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:53.106460 18004 catalog_manager.cc:1347] Generated new cluster ID: ef51060238fd42aabea46e187f48f79b
I20260812 00:06:53.106523 18004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:53.116264 18004 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:53.116772 18004 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:53.124644 18004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9daf0d3a05d411fb54bef12facf43a4: Generated new TSK 0
I20260812 00:06:53.124866 18004 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 00:06:53.149986 18038 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 00:06:53.150089 18038 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 00:06:53.150436 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:53.134542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.7.85.254:46113)
0812 00:06:53.134618 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:53.134621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:53.134848 (+ 227us) server_negotiation.cc:407] Connection header received
0812 00:06:53.134930 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:53.134932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:53.134950 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:53.135069 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:06:53.135328 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.136072 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.137314 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.137539 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.137624 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.137870 (+ 246us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:06:53.150240 (+ 12370us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 00:06:53.150380 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 00:06:53.150512 17989 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51246: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 00:06:53.150648 18037 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:53.134486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46113 (local address 127.0.0.1:51246)
0812 00:06:53.134670 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:53.134688 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:53.134761 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:53.135091 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:53.135092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:53.135094 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:06:53.135156 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:53.135159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.136187 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.136188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:53.137213 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:53.137216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.137646 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.137647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:53.137723 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.137730 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:06:53.150383 (+ 12653us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 00:06:53.150534 (+ 151us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.85.254:46113: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35051/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260812 00:06:53.150977 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17861
2026-08-12T00:06:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 00:06:53.193900 7511 mini_oidc.cc:152] Starting JWKS server
I20260812 00:06:53.195180 7511 webserver.cc:460] Webserver started at https://127.0.0.1:44257/ using document root <none> and password file <none>
I20260812 00:06:53.195307 7511 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 00:06:53.195518 7511 webserver.cc:460] Webserver started at http://127.0.0.1:36401/ using document root <none> and password file <none>
2026-08-12T00:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:53Z Disabled control of system clock
I20260812 00:06:53.207450 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:42329
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:36997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:42329
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44257/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 00:06:53.303517 18047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:53.303759 18047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:53.303784 18047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:53.305248 18047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:53.305302 18047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:53.305316 18047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:53.305332 18047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:53.305344 18047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:06:53.305357 18047 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:06:53.305369 18047 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44257/jwks.json
I20260812 00:06:53.306838 18047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:42329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:42329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44257/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:53.307111 18047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:53.307448 18047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:53.312472 18059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:53.312476 18054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:53.312624 18056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:53.312867 18047 server_base.cc:1034] running on GCE node
I20260812 00:06:53.313323 18047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:53.314147 18047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:53.315344 18047 hybrid_clock.cc:648] HybridClock initialized: now 1786493213315324 us; error 49 us; skew 500 ppm
I20260812 00:06:53.316741 18047 webserver.cc:460] Webserver started at http://127.7.85.254:42705/ using document root <none> and password file <none>
I20260812 00:06:53.317011 18047 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:53.317062 18047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:53.317502 18047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:53.318693 18047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "ac048e0b43f04218a435caf64c43c3be"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "4a906319d48c1a2b852cf899cf9a52f6"
server_key_iv: "64c0bc59bb4861d5268c056b104b2b4c"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.319061 18047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "ac048e0b43f04218a435caf64c43c3be"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "4a906319d48c1a2b852cf899cf9a52f6"
server_key_iv: "64c0bc59bb4861d5268c056b104b2b4c"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.320487 18047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:53.321359 18070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:53.321558 18047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:06:53.321672 18047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "ac048e0b43f04218a435caf64c43c3be"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "4a906319d48c1a2b852cf899cf9a52f6"
server_key_iv: "64c0bc59bb4861d5268c056b104b2b4c"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.321749 18047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:53.337095 18047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:53.338160 18047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:53.338831 18047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:53.344580 18047 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:42329
I20260812 00:06:53.344620 18165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:42329 every 8 connection(s)
I20260812 00:06:53.345062 18047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:53.345884 18166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:53.349192 18166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be: Bootstrap starting.
I20260812 00:06:53.349911 18166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:53.350270 18166 log.cc:824] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:53.351297 18166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be: No bootstrap required, opened a new log
I20260812 00:06:53.353025 18166 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac048e0b43f04218a435caf64c43c3be" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42329 } }
I20260812 00:06:53.353245 18166 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:53.353291 18166 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac048e0b43f04218a435caf64c43c3be, State: Initialized, Role: FOLLOWER
I20260812 00:06:53.353533 18166 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac048e0b43f04218a435caf64c43c3be" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42329 } }
I20260812 00:06:53.353643 18166 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:53.353683 18166 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:53.353742 18166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:53.354379 18166 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac048e0b43f04218a435caf64c43c3be" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42329 } }
I20260812 00:06:53.354511 18166 leader_election.cc:302] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac048e0b43f04218a435caf64c43c3be; no voters:
I20260812 00:06:53.354727 18166 leader_election.cc:288] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:53.354897 18171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:53.355054 18166 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:53.355249 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18047
I20260812 00:06:53.355321 18171 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [term 1 LEADER]: Becoming Leader. State: Replica: ac048e0b43f04218a435caf64c43c3be, State: Running, Role: LEADER
I20260812 00:06:53.355388 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:53.355466 18171 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac048e0b43f04218a435caf64c43c3be" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42329 } }
I20260812 00:06:53.355625 7511 external_mini_cluster.cc:1428] Setting key 60ba4933fea63001af06d2b3e5b078dc
I20260812 00:06:53.356462 18174 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac048e0b43f04218a435caf64c43c3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac048e0b43f04218a435caf64c43c3be" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42329 } } }
I20260812 00:06:53.356575 18174 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:53.356942 18175 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac048e0b43f04218a435caf64c43c3be. Latest consensus state: current_term: 1 leader_uuid: "ac048e0b43f04218a435caf64c43c3be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac048e0b43f04218a435caf64c43c3be" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 42329 } } }
I20260812 00:06:53.357035 18175 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:53.356942 18202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:53.358099 18202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:53.359758 18202 catalog_manager.cc:1347] Generated new cluster ID: 483de590d539487c9af0bc4442a97b54
I20260812 00:06:53.359828 18202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:53.363404 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:53.356222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60188 (local address 127.7.85.254:42329)
0812 00:06:53.356481 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:53.356490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:53.357317 (+ 827us) server_negotiation.cc:407] Connection header received
0812 00:06:53.357528 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:53.357535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:53.357593 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:53.357700 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:53.358093 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.359654 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.360851 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.361701 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.361863 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.362618 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:53.362622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:53.362623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:53.362840 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:53.362883 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:53.362898 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:53.362971 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:53.363037 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 00:06:53.363132 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260812 00:06:53.364471 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:06:53.370734 18202 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:53.371322 18202 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:53.374609 18202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac048e0b43f04218a435caf64c43c3be: Generated new TSK 0
I20260812 00:06:53.374953 18202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:06:53.388080 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:53.383365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60192 (local address 127.7.85.254:42329)
0812 00:06:53.383423 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:53.383425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:53.383477 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:06:53.383591 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:53.383592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:53.383604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:53.383671 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:06:53.383868 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.384494 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.386152 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.386347 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.386410 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.386685 (+ 275us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:06:53.387854 (+ 1169us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44257/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 00:06:53.387998 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60192: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44257/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 00:06:53.388147 18194 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60192: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44257/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 00:06:53.388207 18223 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:53.383238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:42329 (local address 127.0.0.1:60192)
0812 00:06:53.383407 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:53.383420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:53.383500 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:53.383691 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:53.383692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:53.383694 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:06:53.383759 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:53.383761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.384792 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.384793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:53.385914 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:53.385918 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.386438 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.386439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:53.386514 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.386521 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:06:53.387977 (+ 1456us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44257/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 00:06:53.388094 (+ 117us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.85.254:42329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44257/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260812 00:06:53.388679 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18047
E20260812 00:06:53.404752 18230 webserver.cc:533] Webserver: sslize failed:
E20260812 00:06:53.404888 18230 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T00:06:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T00:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:53Z Disabled control of system clock
I20260812 00:06:53.446782 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41473
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41473
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/testchainca.cert with env {}
W20260812 00:06:53.538638 18236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:53.538864 18236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:53.538888 18236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:53.540123 18236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:53.540185 18236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:53.540198 18236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:53.540217 18236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:53.540230 18236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:06:53.540242 18236 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:06:53.540254 18236 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 00:06:53.542032 18236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:53.542330 18236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:53.542564 18236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:53.548377 18243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:53.548493 18253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:53.548377 18251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:53.548661 18236 server_base.cc:1034] running on GCE node
I20260812 00:06:53.548945 18236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:53.549346 18236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:53.550540 18236 hybrid_clock.cc:648] HybridClock initialized: now 1786493213550507 us; error 47 us; skew 500 ppm
I20260812 00:06:53.552686 18236 webserver.cc:460] Webserver started at http://127.7.85.254:45693/ using document root <none> and password file <none>
I20260812 00:06:53.553041 18236 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:53.553105 18236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:53.553303 18236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:53.554584 18236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "5acb44820c97413da4dafa00ad0a25ec"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "649f80dae92faab918406e5fc9382eff"
server_key_iv: "993eabd97e3b2752611f3aae7c36a6e8"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.554931 18236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "5acb44820c97413da4dafa00ad0a25ec"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "649f80dae92faab918406e5fc9382eff"
server_key_iv: "993eabd97e3b2752611f3aae7c36a6e8"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.556442 18236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:06:53.557689 18261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:53.558102 18236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:53.558247 18236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "5acb44820c97413da4dafa00ad0a25ec"
format_stamp: "Formatted at 2026-08-12 00:06:53 on dist-test-slave-z8x9"
server_key: "649f80dae92faab918406e5fc9382eff"
server_key_iv: "993eabd97e3b2752611f3aae7c36a6e8"
server_key_version: "encryptionkey@0"
I20260812 00:06:53.558324 18236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:53.570487 18236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:53.571077 18236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:53.571237 18236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:53.577235 18236 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:41473
I20260812 00:06:53.577271 18357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:41473 every 8 connection(s)
I20260812 00:06:53.577690 18236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:53.578797 18358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:53.581259 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec: Bootstrap starting.
I20260812 00:06:53.581861 18358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:53.582103 18358 log.cc:824] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:53.582515 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18236
I20260812 00:06:53.582638 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:53.582849 7511 external_mini_cluster.cc:1428] Setting key 4eb5aaf0c3058093326a4475e31204d5
I20260812 00:06:53.583019 18358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec: No bootstrap required, opened a new log
I20260812 00:06:53.585302 18358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5acb44820c97413da4dafa00ad0a25ec" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41473 } }
I20260812 00:06:53.585455 18358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:53.585479 18358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acb44820c97413da4dafa00ad0a25ec, State: Initialized, Role: FOLLOWER
I20260812 00:06:53.585638 18358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5acb44820c97413da4dafa00ad0a25ec" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41473 } }
I20260812 00:06:53.585788 18358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:53.585834 18358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:53.585879 18358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:53.586668 18358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5acb44820c97413da4dafa00ad0a25ec" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41473 } }
I20260812 00:06:53.586774 18358 leader_election.cc:302] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5acb44820c97413da4dafa00ad0a25ec; no voters:
I20260812 00:06:53.587199 18358 leader_election.cc:288] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:53.587224 18371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:53.587415 18371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [term 1 LEADER]: Becoming Leader. State: Replica: 5acb44820c97413da4dafa00ad0a25ec, State: Running, Role: LEADER
I20260812 00:06:53.587512 18371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5acb44820c97413da4dafa00ad0a25ec" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41473 } }
I20260812 00:06:53.587528 18358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:53.588017 18371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5acb44820c97413da4dafa00ad0a25ec. Latest consensus state: current_term: 1 leader_uuid: "5acb44820c97413da4dafa00ad0a25ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5acb44820c97413da4dafa00ad0a25ec" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41473 } } }
I20260812 00:06:53.588116 18371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:53.588052 18373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5acb44820c97413da4dafa00ad0a25ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5acb44820c97413da4dafa00ad0a25ec" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41473 } } }
I20260812 00:06:53.588295 18373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:53.588531 18378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:53.589200 18378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:53.591630 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:53.583473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.7.85.254:41473)
0812 00:06:53.583772 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:53.583780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:53.583811 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:06:53.585282 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:53.585292 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:53.585381 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:53.585562 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:53.585903 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.587356 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.588718 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.589550 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.589682 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.590711 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:53.590715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:53.590716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:06:53.590947 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:53.590995 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:06:53.591010 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:53.591084 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:53.591308 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 00:06:53.591395 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260812 00:06:53.591970 18378 catalog_manager.cc:1347] Generated new cluster ID: 82c00b2cf3a64fae9b20c84fe10bf15d
I20260812 00:06:53.592029 18378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:53.592659 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:06:53.606371 18378 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:53.606973 18378 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:53.611034 18378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5acb44820c97413da4dafa00ad0a25ec: Generated new TSK 0
I20260812 00:06:53.611306 18378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:06:54.036614 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:53.618121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.7.85.254:41473)
0812 00:06:53.618211 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:53.618214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:53.618425 (+ 211us) server_negotiation.cc:407] Connection header received
0812 00:06:53.618508 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:53.618510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:53.618525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:53.618616 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:06:53.618847 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.619629 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.620687 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.621028 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.621132 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.621515 (+ 383us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:06:54.036351 (+414836us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 00:06:54.036544 (+ 193us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44150: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 00:06:54.036703 18369 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44150: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 00:06:54.036899 18415 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:53.618012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41473 (local address 127.0.0.1:44150)
0812 00:06:53.618317 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:53.618338 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:53.618419 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:53.618683 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:53.618685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:53.618689 (+ 4us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:06:53.618754 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:53.618757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.619779 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.619782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:53.620568 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:53.620571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:53.621302 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:53.621304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:53.621365 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:53.621374 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:06:54.036555 (+415181us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 00:06:54.036747 (+ 192us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.85.254:41473: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":184,"threads_started":1}
I20260812 00:06:54.037587 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18236
2026-08-12T00:06:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (634 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (100 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (98 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:54 dist-test-slave-z8x9 krb5kdc[18439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:54 dist-test-slave-z8x9 krb5kdc[18439](info): set up 2 sockets
Aug 12 00:06:54 dist-test-slave-z8x9 krb5kdc[18439](info): commencing operation
krb5kdc: starting...
W20260812 00:06:55.630951 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:55 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:06:55.786468 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36343
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:55.884218 18462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:55.884507 18462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:55.884549 18462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:55.885874 18462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:55.885946 18462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:55.885969 18462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:55.885991 18462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:55.886020 18462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:55.887573 18462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:55.887873 18462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:55.888120 18462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:55.894577 18478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:55.894627 18471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:55.894599 18474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:55.895133 18462 server_base.cc:1034] running on GCE node
I20260812 00:06:55.895457 18462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:55.895795 18462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:55.897070 18462 hybrid_clock.cc:648] HybridClock initialized: now 1786493215897031 us; error 57 us; skew 500 ppm
Aug 12 00:06:55 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:55.905328 18462 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:06:55.905857 18462 webserver.cc:460] Webserver started at http://127.7.85.254:38085/ using document root <none> and password file <none>
I20260812 00:06:55.906114 18462 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:55.906163 18462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:55.906313 18462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:55.907472 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "e66a3dace32a4c0e99720f75bd1bdee6"
format_stamp: "Formatted at 2026-08-12 00:06:55 on dist-test-slave-z8x9"
server_key: "4d5b8ddbb70e24b3d8c5e7793f36215f"
server_key_iv: "95d7921f17cfe8b0b585762bfe2db7ab"
server_key_version: "encryptionkey@0"
I20260812 00:06:55.907806 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "e66a3dace32a4c0e99720f75bd1bdee6"
format_stamp: "Formatted at 2026-08-12 00:06:55 on dist-test-slave-z8x9"
server_key: "4d5b8ddbb70e24b3d8c5e7793f36215f"
server_key_iv: "95d7921f17cfe8b0b585762bfe2db7ab"
server_key_version: "encryptionkey@0"
I20260812 00:06:55.909552 18462 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 00:06:55.910516 18487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:55.910835 18462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:55.910970 18462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "e66a3dace32a4c0e99720f75bd1bdee6"
format_stamp: "Formatted at 2026-08-12 00:06:55 on dist-test-slave-z8x9"
server_key: "4d5b8ddbb70e24b3d8c5e7793f36215f"
server_key_iv: "95d7921f17cfe8b0b585762bfe2db7ab"
server_key_version: "encryptionkey@0"
I20260812 00:06:55.911056 18462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:55.934587 18462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:55.938176 18462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:55.938405 18462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:55.943420 18462 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:36343
I20260812 00:06:55.943897 18462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:55.944386 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18462
I20260812 00:06:55.944525 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:55.944479 18587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:36343 every 8 connection(s)
I20260812 00:06:55.944834 7511 external_mini_cluster.cc:1428] Setting key 6771a7f19d240e99f2efcd53151c0b75
I20260812 00:06:55.944883 18588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:55.948525 18588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6: Bootstrap starting.
I20260812 00:06:55.949389 18588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:55.949704 18588 log.cc:824] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:55.950539 18588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6: No bootstrap required, opened a new log
Aug 12 00:06:55 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:55.953418 18588 raft_consensus.cc:348] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36343 } }
I20260812 00:06:55.953624 18588 raft_consensus.cc:374] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:55.953687 18588 raft_consensus.cc:729] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e66a3dace32a4c0e99720f75bd1bdee6, State: Initialized, Role: FOLLOWER
I20260812 00:06:55.953882 18588 consensus_queue.cc:260] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36343 } }
I20260812 00:06:55.953989 18588 raft_consensus.cc:388] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:55.954033 18588 raft_consensus.cc:482] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:55.954084 18588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:55.954782 18588 raft_consensus.cc:504] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36343 } }
I20260812 00:06:55.955128 18588 leader_election.cc:302] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e66a3dace32a4c0e99720f75bd1bdee6; no voters:
I20260812 00:06:55.955621 18588 leader_election.cc:288] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:55.955660 18602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:55.955862 18602 raft_consensus.cc:686] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [term 1 LEADER]: Becoming Leader. State: Replica: e66a3dace32a4c0e99720f75bd1bdee6, State: Running, Role: LEADER
I20260812 00:06:55.956053 18588 sys_catalog.cc:564] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:55.956022 18602 consensus_queue.cc:237] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36343 } }
I20260812 00:06:55.956472 18608 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36343 } } }
I20260812 00:06:55.956723 18608 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:55.956641 18603 sys_catalog.cc:455] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e66a3dace32a4c0e99720f75bd1bdee6. Latest consensus state: current_term: 1 leader_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e66a3dace32a4c0e99720f75bd1bdee6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36343 } } }
I20260812 00:06:55.957026 18603 sys_catalog.cc:458] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:55.957259 18625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:55.957921 18625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:55.960052 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:55.945534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.7.85.254:36343)
0812 00:06:55.945718 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:55.945729 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:55.945763 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:06:55.946162 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:55.946170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:55.946218 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:55.946339 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:55.947245 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:55.948201 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:55.949450 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:55.950161 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:55.950289 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:55.953387 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:55.953393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:55.953393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:55.953408 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:55.957770 (+ 4362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:55.958318 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:55.958322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:55.958322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:55.958459 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:55.958768 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:55.958770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:55.958770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:55.959059 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:55.959256 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:55.959493 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 00:06:55.959665 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260812 00:06:55.960479 18625 catalog_manager.cc:1347] Generated new cluster ID: 658cd5abd5bf4fccb388e4eaa9804e18
I20260812 00:06:55.960546 18625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:06:55.972048 18625 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:55.973277 18625 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:55.984165 18625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e66a3dace32a4c0e99720f75bd1bdee6: Generated new TSK 0
I20260812 00:06:55.984448 18625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:06:56.024855 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36343
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:56.121204 18649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:56.121454 18649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:56.121480 18649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:56.122730 18649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:56.122786 18649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:56.122815 18649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:56.124425 18649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:56.124715 18649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:56.124996 18649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:56.130625 18660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:56.130615 18664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:56.130620 18661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:56.131098 18649 server_base.cc:1034] running on GCE node
I20260812 00:06:56.131342 18649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:56.131588 18649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:56.132769 18649 hybrid_clock.cc:648] HybridClock initialized: now 1786493216132728 us; error 55 us; skew 500 ppm
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.135835 18649 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:06:56.136799 18649 webserver.cc:460] Webserver started at http://127.7.85.193:43685/ using document root <none> and password file <none>
I20260812 00:06:56.137070 18649 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:56.137123 18649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:56.137322 18649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:56.138477 18649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "ee5e8325ded2470eb55fa62d4542e05a"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "77457b9fdf7dd40b9da7160c266d4b04"
server_key_iv: "06a0e5c963e6cfb994af1de72179cadf"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.138857 18649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "ee5e8325ded2470eb55fa62d4542e05a"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "77457b9fdf7dd40b9da7160c266d4b04"
server_key_iv: "06a0e5c963e6cfb994af1de72179cadf"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.140259 18649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:56.141103 18676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.141352 18649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:56.141441 18649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "ee5e8325ded2470eb55fa62d4542e05a"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "77457b9fdf7dd40b9da7160c266d4b04"
server_key_iv: "06a0e5c963e6cfb994af1de72179cadf"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.141497 18649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:56.157478 18649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:56.160101 18649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:56.160233 18649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:56.160662 18649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:56.161473 18649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:56.161614 18649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.161669 18649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:56.161702 18649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.169703 18649 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:41067
I20260812 00:06:56.169790 18893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:41067 every 8 connection(s)
I20260812 00:06:56.170168 18649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:56.171835 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18649
I20260812 00:06:56.171967 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:56.172254 7511 external_mini_cluster.cc:1428] Setting key 5d6f51b5f557fe21b78d3c260c47612e
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:06:56.182668 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.171096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:55039 (local address 127.7.85.254:36343)
0812 00:06:56.171192 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.171195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.171314 (+ 119us) server_negotiation.cc:407] Connection header received
0812 00:06:56.172692 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.172695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.172719 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.172818 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.173831 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.174379 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.175717 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.176085 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.176173 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.179029 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:56.179031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:56.179032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:56.179042 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:56.181029 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.181546 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.181548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.181549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.181624 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.181913 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.181914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.181915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.182048 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:56.182123 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:56.182438 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 00:06:56.182560 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:06:56.182871 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.170996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36343 (local address 127.7.85.193:55039)
0812 00:06:56.171176 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:56.171237 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:56.172534 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:56.172912 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:56.172916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:56.173454 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:56.173686 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:56.173691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.174488 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.174490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:56.175584 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:56.175587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.176199 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.176200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:56.176377 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.177014 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:56.177026 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:56.178915 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:56.181147 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:56.181157 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:56.181160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:56.181460 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:56.181698 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:56.181699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:56.181700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:56.181832 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:56.182125 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:56.182134 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:56.182290 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 00:06:56.182402 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 00:06:56.182591 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260812 00:06:56.183715 18898 heartbeater.cc:347] Connected to a master server at 127.7.85.254:36343
I20260812 00:06:56.183852 18898 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:56.184062 18898 heartbeater.cc:511] Master 127.7.85.254:36343 requested a full tablet report, sending...
I20260812 00:06:56.184679 18525 ts_manager.cc:194] Registered new tserver with Master: ee5e8325ded2470eb55fa62d4542e05a (127.7.85.193:41067)
I20260812 00:06:56.186074 18525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:55039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:06:56.222466 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36343
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:56.308881 18907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:56.309199 18907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:56.309276 18907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:56.310508 18907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:56.310586 18907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:56.310672 18907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:56.312294 18907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:56.312593 18907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:56.312865 18907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:56.318718 18915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:56.318770 18914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:56.318734 18918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:56.318986 18907 server_base.cc:1034] running on GCE node
I20260812 00:06:56.319267 18907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:56.319514 18907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:56.320706 18907 hybrid_clock.cc:648] HybridClock initialized: now 1786493216320656 us; error 45 us; skew 500 ppm
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.323660 18907 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:06:56.324160 18907 webserver.cc:460] Webserver started at http://127.7.85.194:38031/ using document root <none> and password file <none>
I20260812 00:06:56.324435 18907 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:56.324510 18907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:56.324667 18907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:56.325824 18907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "9d245e47a6dc4f05a2c6fafbb0ba2eda"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "48ab37f78e8ef36df3d7ea1c960cca73"
server_key_iv: "6f09217e925b8469e4b3ebbffe3c89b4"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.326203 18907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "9d245e47a6dc4f05a2c6fafbb0ba2eda"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "48ab37f78e8ef36df3d7ea1c960cca73"
server_key_iv: "6f09217e925b8469e4b3ebbffe3c89b4"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.327630 18907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:06:56.328469 18934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.328694 18907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:56.328835 18907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "9d245e47a6dc4f05a2c6fafbb0ba2eda"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "48ab37f78e8ef36df3d7ea1c960cca73"
server_key_iv: "6f09217e925b8469e4b3ebbffe3c89b4"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.328918 18907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:56.352941 18907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:56.355607 18907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:56.355780 18907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:56.356047 18907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:56.356427 18907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:56.356493 18907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.356629 18907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:56.356685 18907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.365499 18907 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:46137
I20260812 00:06:56.365543 19132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:46137 every 8 connection(s)
I20260812 00:06:56.365907 18907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:06:56.370148 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18907
I20260812 00:06:56.370246 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:56.370476 7511 external_mini_cluster.cc:1428] Setting key 62811ddda4a4d947d9fdc036bc26e059
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:56.378587 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.366834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:38839 (local address 127.7.85.254:36343)
0812 00:06:56.366891 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.366893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.367176 (+ 283us) server_negotiation.cc:407] Connection header received
0812 00:06:56.368194 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.368196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.368212 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.368282 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.369032 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.369656 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.371472 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.371680 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.371773 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.374393 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:56.374394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:56.374395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:56.374402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:56.376735 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.377292 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.377293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.377294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.377370 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.377719 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.377721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.377721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.377899 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:56.377990 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:56.378349 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 00:06:56.378460 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 00:06:56.378857 19136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.366705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36343 (local address 127.7.85.194:38839)
0812 00:06:56.367002 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:56.367066 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:56.368065 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:56.368400 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:56.368404 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:56.368732 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:56.368936 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:56.368940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.369849 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.369852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:56.371356 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:56.371362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.371787 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.371788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:56.371932 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.372540 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:56.372549 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:56.374243 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:56.376861 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:56.376869 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:56.376872 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:56.377147 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:56.377481 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:56.377481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:56.377482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:56.377624 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:56.378022 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:56.378029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:56.378188 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 00:06:56.378293 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 00:06:56.378447 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":113,"threads_started":1}
I20260812 00:06:56.379501 19133 heartbeater.cc:347] Connected to a master server at 127.7.85.254:36343
I20260812 00:06:56.379626 19133 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:56.379848 19133 heartbeater.cc:511] Master 127.7.85.254:36343 requested a full tablet report, sending...
I20260812 00:06:56.380410 18525 ts_manager.cc:194] Registered new tserver with Master: 9d245e47a6dc4f05a2c6fafbb0ba2eda (127.7.85.194:46137)
I20260812 00:06:56.380982 18525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:38839
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:06:56.405236 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36343
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:56.502285 19144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:56.502534 19144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:56.502562 19144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:56.504024 19144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:56.504091 19144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:56.504122 19144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:56.505887 19144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:56.506122 19144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:56.506336 19144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:56.512454 19150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:56.512671 19144 server_base.cc:1034] running on GCE node
W20260812 00:06:56.512686 19152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:56.512686 19156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:56.513147 19144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:56.513540 19144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:56.514876 19144 hybrid_clock.cc:648] HybridClock initialized: now 1786493216514841 us; error 81 us; skew 500 ppm
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.517951 19144 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:06:56.518499 19144 webserver.cc:460] Webserver started at http://127.7.85.195:46467/ using document root <none> and password file <none>
I20260812 00:06:56.518752 19144 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:56.518826 19144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:56.518989 19144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:56.520277 19144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "68446ca4e6dc4968ba7ab121ba7bd146"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "9e637952418940f16355928b84143820"
server_key_iv: "737ca91786c10df8bd7a3f0449806a4c"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.520684 19144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "68446ca4e6dc4968ba7ab121ba7bd146"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "9e637952418940f16355928b84143820"
server_key_iv: "737ca91786c10df8bd7a3f0449806a4c"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.522322 19144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:06:56.523280 19166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.523545 19144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:56.523674 19144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "68446ca4e6dc4968ba7ab121ba7bd146"
format_stamp: "Formatted at 2026-08-12 00:06:56 on dist-test-slave-z8x9"
server_key: "9e637952418940f16355928b84143820"
server_key_iv: "737ca91786c10df8bd7a3f0449806a4c"
server_key_version: "encryptionkey@0"
I20260812 00:06:56.523768 19144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:56.532588 19144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:56.535283 19144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:56.535477 19144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:56.535739 19144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:56.536070 19144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:56.536139 19144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.536180 19144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:56.536209 19144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:56.542565 19144 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:42367
I20260812 00:06:56.542655 19373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:42367 every 8 connection(s)
I20260812 00:06:56.542989 19144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:06:56.552322 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 19144
I20260812 00:06:56.552438 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:56.552661 7511 external_mini_cluster.cc:1428] Setting key b44953786ba36adb497fb8a1ae3e120a
I20260812 00:06:56.553279 18592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.543880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:53941 (local address 127.7.85.254:36343)
0812 00:06:56.543944 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.543946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.544145 (+ 199us) server_negotiation.cc:407] Connection header received
0812 00:06:56.545138 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.545140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.545151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.545271 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.545961 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.546510 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.547522 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.547717 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.547774 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.550114 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:56.550116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:56.550117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:56.550126 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:56.551480 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.551937 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.551938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.551939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.552009 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.552435 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.552437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.552437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.552590 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:56.552657 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:56.553052 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 00:06:56.553144 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 00:06:56.553522 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.543757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36343 (local address 127.7.85.195:53941)
0812 00:06:56.543984 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:56.544028 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:56.544953 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:56.545361 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:56.545365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:56.545660 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:56.545872 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:56.545876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.546605 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.546607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:56.547439 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:56.547441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.547810 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.547811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:56.547919 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.548460 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:56.548469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:56.550003 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:56.551578 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:56.551586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:56.551589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:56.551849 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:56.552096 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:56.552097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:56.552097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:56.552266 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:56.552671 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:56.552679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:56.552862 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 00:06:56.553009 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 00:06:56.553213 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260812 00:06:56.553997 19375 heartbeater.cc:347] Connected to a master server at 127.7.85.254:36343
I20260812 00:06:56.554126 19375 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:56.554322 19375 heartbeater.cc:511] Master 127.7.85.254:36343 requested a full tablet report, sending...
I20260812 00:06:56.554836 18525 ts_manager.cc:194] Registered new tserver with Master: 68446ca4e6dc4968ba7ab121ba7bd146 (127.7.85.195:42367)
I20260812 00:06:56.555243 18525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:53941
I20260812 00:06:56.565421 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/test-admin.keytab.
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.587052 7511 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 00:06:56.588855 19385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:56.588032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.7.85.193:41067)
0812 00:06:56.588204 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.588209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.588218 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:56.588244 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.588245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.588261 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.588321 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.588751 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49390: BlockingRecv error: recv got EOF from 127.0.0.1:49390 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
W20260812 00:06:56.588977 7511 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc)
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.590200 7511 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.591292 7511 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:56.601449 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.591933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.7.85.193:41067)
0812 00:06:56.591982 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.591983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.592008 (+ 25us) server_negotiation.cc:407] Connection header received
0812 00:06:56.592081 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.592082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.592093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.592144 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.592619 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.593627 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.594490 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.594865 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.594936 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.598045 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:56.598056 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:56.598056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:56.598064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:56.599679 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.600100 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.600104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.600104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.600171 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.600459 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.600460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.600461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.600647 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:56.600811 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:56.601050 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 00:06:56.601212 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 00:06:56.603945 19385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:56.603323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.7.85.193:41067)
0812 00:06:56.603391 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.603393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.603402 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:56.603430 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.603430 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.603440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.603494 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.603905 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49418: BlockingRecv error: recv got EOF from 127.0.0.1:49418 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 00:06:56.604053 7511 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc)
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.605485 7511 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.606685 7511 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:56.615975 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.607459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.7.85.193:41067)
0812 00:06:56.607523 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.607524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.607532 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:06:56.607566 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.607567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.607578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.607628 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.608082 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.608591 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.609724 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.609950 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.610027 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.612829 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:56.612830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:56.612831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:56.612838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:56.614431 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.614894 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.614895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.614895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.614953 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.615178 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.615179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.615179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.615286 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:56.615399 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:56.615824 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 00:06:56.615904 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 00:06:56.618286 19385 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:06:56.617324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.7.85.193:41067)
0812 00:06:56.617391 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.617393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.617481 (+ 88us) server_negotiation.cc:407] Connection header received
0812 00:06:56.617546 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.617547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.617556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.617619 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.618183 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49434: BlockingRecv error: recv got EOF from 127.0.0.1:49434 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 00:06:56.618325 7511 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786493185917310-7511-0/krb5kdc/krb5cc' of type FILE
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.619529 7511 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:56.620460 7511 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.193@KRBTEST.COM
I20260812 00:06:56.628918 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:56.621159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.7.85.193:41067)
0812 00:06:56.621239 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:56.621241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:56.621390 (+ 149us) server_negotiation.cc:407] Connection header received
0812 00:06:56.621453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:56.621454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:56.621464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:56.621530 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:56.621948 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.622512 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.623440 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:56.623642 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:56.623720 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:56.626178 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:56.626179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:56.626180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:56.626186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:56.627679 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.628109 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.628110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.628110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.628175 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:56.628424 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:56.628425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:56.628425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:56.628533 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:56.628626 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:56.628777 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 00:06:56.628844 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:06:56.629900 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18649
I20260812 00:06:56.650377 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18907
I20260812 00:06:56.666505 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 19144
I20260812 00:06:56.687140 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18462
2026-08-12T00:06:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2465 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[19407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[19407](info): set up 2 sockets
Aug 12 00:06:56 dist-test-slave-z8x9 krb5kdc[19407](info): commencing operation
krb5kdc: starting...
W20260812 00:06:58.075160 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:06:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab.
Entry for principal oryx/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab.
I20260812 00:06:58.217547 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:43135
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab
--principal=oryx/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:43135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:58.308832 19429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:58.309113 19429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:58.309155 19429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:58.310345 19429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:58.310418 19429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:58.310444 19429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:58.310472 19429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:58.310504 19429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:58.312034 19429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:43135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab
--principal=oryx/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:43135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:58.312338 19429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:58.312582 19429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:58.318152 19448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.318152 19444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.318142 19443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:58.319416 19429 server_base.cc:1034] running on GCE node
I20260812 00:06:58.319742 19429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:58.320066 19429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:58.321282 19429 hybrid_clock.cc:648] HybridClock initialized: now 1786493218321258 us; error 39 us; skew 500 ppm
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:58.324592 19429 init.cc:377] Logged in from keytab as oryx/127.7.85.254@KRBTEST.COM (short username oryx)
I20260812 00:06:58.325112 19429 webserver.cc:460] Webserver started at http://127.7.85.254:39849/ using document root <none> and password file <none>
I20260812 00:06:58.325412 19429 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:58.325488 19429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:58.325647 19429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:58.326774 19429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "c79758a37f514ac292fd19a499799159"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "dc42339f937fbe9031d9b0413df78dd9"
server_key_iv: "e825e22b48cdeb8ed9a800add5cd859f"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.327152 19429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "c79758a37f514ac292fd19a499799159"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "dc42339f937fbe9031d9b0413df78dd9"
server_key_iv: "e825e22b48cdeb8ed9a800add5cd859f"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.328620 19429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:06:58.329905 19463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.330159 19429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:58.330287 19429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "c79758a37f514ac292fd19a499799159"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "dc42339f937fbe9031d9b0413df78dd9"
server_key_iv: "e825e22b48cdeb8ed9a800add5cd859f"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.330367 19429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:58.342573 19429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:58.345316 19429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:58.345491 19429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:58.349619 19429 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:43135
I20260812 00:06:58.349655 19568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:43135 every 8 connection(s)
I20260812 00:06:58.350052 19429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:06:58.351104 19570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:58.353592 19570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Bootstrap starting.
I20260812 00:06:58.354197 19570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:58.354833 19570 log.cc:824] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:58.355170 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 19429
I20260812 00:06:58.355350 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:06:58.355589 7511 external_mini_cluster.cc:1428] Setting key f66819b5b95594ba1bf39a6b17dda7f3
I20260812 00:06:58.355930 19570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: No bootstrap required, opened a new log
I20260812 00:06:58.357906 19570 raft_consensus.cc:348] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } }
I20260812 00:06:58.358078 19570 raft_consensus.cc:374] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:58.358125 19570 raft_consensus.cc:729] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79758a37f514ac292fd19a499799159, State: Initialized, Role: FOLLOWER
I20260812 00:06:58.358284 19570 consensus_queue.cc:260] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } }
I20260812 00:06:58.358377 19570 raft_consensus.cc:388] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:06:58.358422 19570 raft_consensus.cc:482] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:06:58.358495 19570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:58.359447 19570 raft_consensus.cc:504] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } }
I20260812 00:06:58.359578 19570 leader_election.cc:302] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c79758a37f514ac292fd19a499799159; no voters:
I20260812 00:06:58.359766 19570 leader_election.cc:288] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:06:58.359858 19577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:06:58.360002 19577 raft_consensus.cc:686] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 LEADER]: Becoming Leader. State: Replica: c79758a37f514ac292fd19a499799159, State: Running, Role: LEADER
I20260812 00:06:58.360065 19570 sys_catalog.cc:564] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:58.360090 19577 consensus_queue.cc:237] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } }
I20260812 00:06:58.360451 19577 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c79758a37f514ac292fd19a499799159. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } }
I20260812 00:06:58.360555 19577 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:58.360471 19578 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } }
I20260812 00:06:58.360761 19578 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:58.360891 19586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:06:58.361501 19586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:06:58.363294 19586 catalog_manager.cc:1347] Generated new cluster ID: bca890f5cf0f40b0a221b08e88d5662f
I20260812 00:06:58.363351 19586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:58.369362 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.356111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56932 (local address 127.7.85.254:43135)
0812 00:06:58.356311 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:58.356319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:58.356345 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:06:58.357926 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:58.357936 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:58.357994 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:58.358177 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:58.358955 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.360242 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.361530 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.362248 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.362363 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.365217 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:58.365222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:58.365222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:58.365236 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:58.367546 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.367971 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.367975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.367975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.368119 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.368337 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.368338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.368338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.368698 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:58.368898 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:58.368969 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 00:06:58.369120 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260812 00:06:58.383741 19586 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:06:58.384323 19586 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:06:58.389846 19586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Generated new TSK 0
I20260812 00:06:58.390053 19586 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab.
Entry for principal oryx/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab.
I20260812 00:06:58.430437 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43135
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:58.522550 19610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:58.522938 19610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:58.523015 19610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:58.524672 19610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:58.524724 19610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:58.524752 19610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:58.526553 19610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:58.526991 19610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:58.527311 19610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:58.532770 19618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.532761 19619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.532754 19622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:58.533154 19610 server_base.cc:1034] running on GCE node
I20260812 00:06:58.533655 19610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:58.533951 19610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:58.535167 19610 hybrid_clock.cc:648] HybridClock initialized: now 1786493218535145 us; error 42 us; skew 500 ppm
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:58.538358 19610 init.cc:377] Logged in from keytab as oryx/127.7.85.193@KRBTEST.COM (short username oryx)
I20260812 00:06:58.538836 19610 webserver.cc:460] Webserver started at http://127.7.85.193:41955/ using document root <none> and password file <none>
I20260812 00:06:58.539075 19610 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:58.539145 19610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:58.539307 19610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:58.540573 19610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "330181aaf8134023a96c5e42e73e08b0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "230c93c10915935b4f8a124562544f14"
server_key_iv: "76b77ebe3ee0f2ca571e83ea12a15d58"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.540969 19610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "330181aaf8134023a96c5e42e73e08b0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "230c93c10915935b4f8a124562544f14"
server_key_iv: "76b77ebe3ee0f2ca571e83ea12a15d58"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.542743 19610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:58.543629 19636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.543919 19610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:58.544039 19610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "330181aaf8134023a96c5e42e73e08b0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "230c93c10915935b4f8a124562544f14"
server_key_iv: "76b77ebe3ee0f2ca571e83ea12a15d58"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.544117 19610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:58.554344 19610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:58.556849 19610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:58.557046 19610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:58.557395 19610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:58.557782 19610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:58.557842 19610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.557940 19610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:58.558007 19610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.565487 19610 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:40603
I20260812 00:06:58.565558 19869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:40603 every 8 connection(s)
I20260812 00:06:58.565919 19610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:58.568311 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 19610
I20260812 00:06:58.568450 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:06:58.568698 7511 external_mini_cluster.cc:1428] Setting key 0926b9eb233fb97165a0386f487e653e
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:58.578461 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.566680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:49533)
0812 00:06:58.566922 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:58.566977 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:58.567746 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:58.568049 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:58.568053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:58.569028 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:58.569354 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:58.569359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.570079 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.570080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:58.570974 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:58.570977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.571385 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.571387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:58.571540 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.572238 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:58.572250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:58.573881 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:58.576391 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:58.576402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:58.576405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:58.576913 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:58.577274 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:58.577276 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:58.577276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:58.577396 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:58.577763 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:58.577770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:58.577919 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 00:06:58.578026 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 00:06:58.578183 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":113,"threads_started":1}
I20260812 00:06:58.578775 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.566796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:49533 (local address 127.7.85.254:43135)
0812 00:06:58.566898 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:58.566901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:58.567065 (+ 164us) server_negotiation.cc:407] Connection header received
0812 00:06:58.567882 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:58.567883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:58.567901 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:58.567968 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:58.569469 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.569975 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.571075 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.571295 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.571376 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.573992 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:58.573995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:58.573995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:58.574007 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:58.576287 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.577041 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.577043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.577043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.577127 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.577476 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.577477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.577478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.577656 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:58.577750 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:58.578084 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 00:06:58.578192 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 00:06:58.579383 19871 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43135
I20260812 00:06:58.579501 19871 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:58.579725 19871 heartbeater.cc:511] Master 127.7.85.254:43135 requested a full tablet report, sending...
I20260812 00:06:58.580410 19487 ts_manager.cc:194] Registered new tserver with Master: 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603)
WARNING: no policy specified for oryx/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:06:58.581151 19487 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:49533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab.
Entry for principal oryx/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab.
I20260812 00:06:58.606101 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43135
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:58.695760 19878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:58.695981 19878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:58.696005 19878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:58.697193 19878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:58.697234 19878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:58.697261 19878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:58.698776 19878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:58.699045 19878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:58.699296 19878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:58.704629 19891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.704653 19885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.704644 19888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:58.704870 19878 server_base.cc:1034] running on GCE node
I20260812 00:06:58.705101 19878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:58.705384 19878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:58.706552 19878 hybrid_clock.cc:648] HybridClock initialized: now 1786493218706527 us; error 39 us; skew 500 ppm
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:58.709301 19878 init.cc:377] Logged in from keytab as oryx/127.7.85.194@KRBTEST.COM (short username oryx)
I20260812 00:06:58.709764 19878 webserver.cc:460] Webserver started at http://127.7.85.194:36549/ using document root <none> and password file <none>
I20260812 00:06:58.710021 19878 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:58.710067 19878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:58.710237 19878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:58.711295 19878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "b72df70ea65b44d9bbcef5d76e3983db"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "9fe798a5de4d86d1c6f5347d35bc8c11"
server_key_iv: "dd19cd716b6433962800e6bb8a7d1819"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.711671 19878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "b72df70ea65b44d9bbcef5d76e3983db"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "9fe798a5de4d86d1c6f5347d35bc8c11"
server_key_iv: "dd19cd716b6433962800e6bb8a7d1819"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.713039 19878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:58.713922 19901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.714124 19878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:58.714210 19878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "b72df70ea65b44d9bbcef5d76e3983db"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "9fe798a5de4d86d1c6f5347d35bc8c11"
server_key_iv: "dd19cd716b6433962800e6bb8a7d1819"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.714314 19878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:58.737942 19878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:58.740240 19878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:58.740384 19878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:58.740655 19878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:58.740989 19878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:58.741025 19878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.741058 19878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:58.741109 19878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.747244 19878 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:46105
I20260812 00:06:58.747327 20111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:46105 every 8 connection(s)
I20260812 00:06:58.747639 19878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:06:58.752707 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 19878
I20260812 00:06:58.752848 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:06:58.753054 7511 external_mini_cluster.cc:1428] Setting key b5cdb28ff467acfbecdf1e571f96a63b
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:58.758698 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.748422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:53231)
0812 00:06:58.748604 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:58.748648 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:58.749214 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:58.749459 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:58.749462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:58.750072 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:58.750266 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:58.750270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.751021 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.751022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:58.751883 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:58.751885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.752266 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.752267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:58.752399 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.753042 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:58.753052 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:58.754948 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:58.756988 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:58.757002 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:58.757005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:58.757391 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:58.757668 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:58.757669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:58.757669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:58.757801 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:58.758127 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:58.758133 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:58.758288 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 00:06:58.758388 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 00:06:58.758523 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260812 00:06:58.758821 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.748617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53231 (local address 127.7.85.254:43135)
0812 00:06:58.748686 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:58.748688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:58.748734 (+ 46us) server_negotiation.cc:407] Connection header received
0812 00:06:58.749329 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:58.749330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:58.749342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:58.749404 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:58.750391 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.750928 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.751975 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.752169 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.752248 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.755053 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:58.755055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:58.755055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:58.755063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:58.756865 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.757474 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.757476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.757476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.757547 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.757889 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.757890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.757890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.758054 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:58.758126 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:58.758552 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 00:06:58.758674 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 00:06:58.759262 20112 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43135
I20260812 00:06:58.759370 20112 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:58.759560 20112 heartbeater.cc:511] Master 127.7.85.254:43135 requested a full tablet report, sending...
I20260812 00:06:58.760062 19487 ts_manager.cc:194] Registered new tserver with Master: b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194:46105)
I20260812 00:06:58.760658 19487 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:53231
WARNING: no policy specified for oryx/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab.
Entry for principal oryx/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab.
I20260812 00:06:58.785386 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43135
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:58.872943 20137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:58.873292 20137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:58.873329 20137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:58.874630 20137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:58.874683 20137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:58.874711 20137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:06:58.876297 20137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:58.876591 20137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:58.876857 20137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:58.882700 20147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:58.882655 20149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:58.882655 20137 server_base.cc:1034] running on GCE node
W20260812 00:06:58.882655 20153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:58.883446 20137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:58.883749 20137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:58.885027 20137 hybrid_clock.cc:648] HybridClock initialized: now 1786493218884982 us; error 57 us; skew 500 ppm
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:58.887950 20137 init.cc:377] Logged in from keytab as oryx/127.7.85.195@KRBTEST.COM (short username oryx)
I20260812 00:06:58.888397 20137 webserver.cc:460] Webserver started at http://127.7.85.195:32969/ using document root <none> and password file <none>
I20260812 00:06:58.888664 20137 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:58.888741 20137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:58.888916 20137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:58.890131 20137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "7c554c26985f30bacc6e2626607b804d"
server_key_iv: "59b65350250824df92161199036ba488"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.890528 20137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "7c554c26985f30bacc6e2626607b804d"
server_key_iv: "59b65350250824df92161199036ba488"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.892132 20137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:06:58.893059 20170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.893316 20137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:58.893440 20137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "7c554c26985f30bacc6e2626607b804d"
server_key_iv: "59b65350250824df92161199036ba488"
server_key_version: "encryptionkey@0"
I20260812 00:06:58.893527 20137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:58.928421 20137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:58.931123 20137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:58.931607 20137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:58.931943 20137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:58.932368 20137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:58.932432 20137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.932479 20137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:58.932509 20137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:58.939280 20137 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:36111
I20260812 00:06:58.939524 20388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:36111 every 8 connection(s)
I20260812 00:06:58.939801 20137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:06:58.942457 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 20137
I20260812 00:06:58.942539 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:06:58.942762 7511 external_mini_cluster.cc:1428] Setting key 567f660cb2751a90e6440c0c4a51aa67
Aug 12 00:06:58 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:58.951962 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.941030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:42855 (local address 127.7.85.254:43135)
0812 00:06:58.941229 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:58.941237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:58.941276 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:06:58.941838 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:58.941840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:58.941855 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:58.941929 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:58.943149 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.944059 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.945347 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.945542 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.945617 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.948195 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:58.948197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:58.948197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:58.948206 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:58.950133 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.950696 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.950700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.950700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.950829 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:58.951213 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:58.951215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:58.951215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:58.951368 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:58.951436 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:58.951800 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 00:06:58.951885 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:06:58.952646 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:58.940896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:42855)
0812 00:06:58.941099 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:58.941153 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:58.941715 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:58.941987 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:58.941990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:58.942699 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:58.942995 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:58.943003 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.944197 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.944198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:58.945253 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:58.945256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:58.945707 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:58.945708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:58.945820 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:58.946459 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:58.946467 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:58.948065 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:58.950247 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:58.950254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:58.950257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:58.950542 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:58.950954 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:58.950955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:58.950955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:58.951097 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:58.951461 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:58.951468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:58.951620 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 00:06:58.951722 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 00:06:58.952295 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260812 00:06:58.953469 20390 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43135
I20260812 00:06:58.953668 20390 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:58.953954 20390 heartbeater.cc:511] Master 127.7.85.254:43135 requested a full tablet report, sending...
I20260812 00:06:58.954818 19487 ts_manager.cc:194] Registered new tserver with Master: 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:06:58.955384 19487 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:42855
I20260812 00:06:58.955755 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.85.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.253.keytab.
Entry for principal oryx/127.7.85.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.253.keytab.
I20260812 00:06:58.990083 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:43829
--webserver_interface=127.7.85.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.253.keytab
--principal=oryx/127.7.85.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:43135,127.7.85.253:43829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:59.087155 20397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:59.087399 20397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:59.087425 20397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:59.088625 20397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:59.088667 20397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:59.088681 20397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:59.088696 20397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:59.088707 20397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:59.090385 20397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:43135,127.7.85.253:43829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.253.keytab
--principal=oryx/127.7.85.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:43829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.85.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
W20260812 00:06:59.090507 20397 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.85.254:43135,127.7.85.253:43829'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 00:06:59.090629 20397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:59.090818 20397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:59.096031 20404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.096040 20403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.096614 20397 server_base.cc:1034] running on GCE node
W20260812 00:06:59.096072 20410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.097620 20397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:59.097996 20397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:59.099221 20397 hybrid_clock.cc:648] HybridClock initialized: now 1786493219099184 us; error 63 us; skew 500 ppm
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:59.102514 20397 init.cc:377] Logged in from keytab as oryx/127.7.85.253@KRBTEST.COM (short username oryx)
I20260812 00:06:59.103121 20397 webserver.cc:460] Webserver started at http://127.7.85.253:33763/ using document root <none> and password file <none>
I20260812 00:06:59.103368 20397 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.103411 20397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.103509 20397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:59.104573 20397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/instance:
uuid: "13529a78b39e49808ed2fc0c855410f7"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "8b455fb45b9d70342ad8c958ec690fef"
server_key_iv: "cbab7d33918087405bb4f6535a6b0cae"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.104902 20397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal/instance:
uuid: "13529a78b39e49808ed2fc0c855410f7"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "8b455fb45b9d70342ad8c958ec690fef"
server_key_iv: "cbab7d33918087405bb4f6535a6b0cae"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.106487 20397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:06:59.107345 20422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.107594 20397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:06:59.107715 20397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
uuid: "13529a78b39e49808ed2fc0c855410f7"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "8b455fb45b9d70342ad8c958ec690fef"
server_key_iv: "cbab7d33918087405bb4f6535a6b0cae"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.107800 20397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.122220 20397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:59.125138 20397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:59.125445 20397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:59.129761 20397 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.253:43829
I20260812 00:06:59.129793 20521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.253:43829 every 8 connection(s)
I20260812 00:06:59.130172 20397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
I20260812 00:06:59.131310 20523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:59.132884 20523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:06:59.138185 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 20397
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.253@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:59.143823 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.133730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.7.85.254:43135)
0812 00:06:59.133851 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.133853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.133983 (+ 130us) server_negotiation.cc:407] Connection header received
0812 00:06:59.134520 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.134521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.134533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.134600 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.135209 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.135726 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.137309 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.137503 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.137575 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.140656 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.140658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.140658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.140684 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.142283 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.142725 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.142726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.142726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.142801 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.143085 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.143085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.143086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.143250 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.143327 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.143633 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.143727 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 00:06:59.144009 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.133641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:56936)
0812 00:06:59.133824 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.133868 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.134387 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.134675 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.134678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.134935 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.135132 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.135135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.135973 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.135976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.137147 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.137152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.137619 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.137621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.137743 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.138351 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.138360 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.140551 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.142392 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.142399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.142402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.142639 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.142891 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.142892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.142892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.143008 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.143331 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.143338 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.143491 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.143599 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.143766 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":129,"threads_started":1}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
I20260812 00:06:59.144737 20523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:06:59.148080 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.138997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.7.85.253:43829)
0812 00:06:59.139207 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.139212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.139239 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:06:59.139265 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.139268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.139290 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.139369 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.140202 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.141031 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.141851 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.142156 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.142258 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.144641 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.144650 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.144650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.144658 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.146599 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.147066 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.147070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.147071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.147138 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.147420 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.147421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.147421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.147605 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.147813 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.147853 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.147957 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":107,"threads_started":1}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.253@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
I20260812 00:06:59.153378 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.145019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55172)
0812 00:06:59.145141 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.145153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.145225 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.145455 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.145455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.145582 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.145624 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.145626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.146370 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.146371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.146983 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.146985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.147525 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.147526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.147602 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.148038 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.148042 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.149450 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.151992 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.151993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.151994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.152187 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.152664 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.152665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.152665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.152729 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.153043 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.153044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.153084 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.153137 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.153305 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260812 00:06:59.153570 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.145136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.7.85.253:43829)
0812 00:06:59.145326 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.145328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.145334 (+ 6us) server_negotiation.cc:407] Connection header received
0812 00:06:59.145352 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.145352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.145363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.145432 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.145685 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.146261 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.147107 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.147439 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.147514 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.149556 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.149557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.149557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.149565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.151814 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.152313 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.152315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.152316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.152413 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.152823 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.152825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.152826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.152971 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.153042 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.153155 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.153374 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":168,"thread_start_us":133,"threads_started":1}
I20260812 00:06:59.155222 20523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Bootstrap starting.
I20260812 00:06:59.155964 20523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:59.156282 20523 log.cc:824] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:59.157392 20523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: No bootstrap required, opened a new log
I20260812 00:06:59.158826 20523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } }
I20260812 00:06:59.158954 20523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:59.158977 20523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13529a78b39e49808ed2fc0c855410f7, State: Initialized, Role: FOLLOWER
I20260812 00:06:59.159129 20523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } }
I20260812 00:06:59.159475 20523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:59.159556 20531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } }
I20260812 00:06:59.159677 20531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:06:59.161883 20558 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:06:59.161965 20558 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:06:59.164997 20397 master_runner.cc:428] Detected that this master 13529a78b39e49808ed2fc0c855410f7 is joining an existing cluster
I20260812 00:06:59.165113 20397 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.85.253:43829
I20260812 00:06:59.173877 20570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.166295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:56944)
0812 00:06:59.166450 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.166462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.166536 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.166718 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.166719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.166913 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.166979 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.166981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.167955 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.167956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.168579 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.168581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.169105 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.169107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.169159 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.169888 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.169893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.170246 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.172184 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.172185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.172185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.172409 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.172823 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.172824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.172824 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.172894 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.173395 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.173396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.173459 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.173546 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.173711 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260812 00:06:59.173943 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.166338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.7.85.254:43135)
0812 00:06:59.166402 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.166405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.166521 (+ 116us) server_negotiation.cc:407] Connection header received
0812 00:06:59.166631 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.166632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.166645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.166715 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.167056 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.167847 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.168671 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.168988 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.169068 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.170350 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.170352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.170352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.170360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.172080 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.172494 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.172495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.172496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.172564 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.172985 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.172986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.172987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.173211 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.173379 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.173750 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.173845 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 00:06:59.180804 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.166317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55178)
0812 00:06:59.173463 (+ 7146us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.173476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.173542 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.173717 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.173718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.173928 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.174540 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.174542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.175271 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.175272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.175971 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.175974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.176781 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.176782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.176859 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.177347 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.177353 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.177814 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.179670 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.179671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.179671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.179844 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.180074 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.180075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.180076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.180134 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.180423 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.180424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.180475 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.180547 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.180659 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7124,"thread_start_us":83,"threads_started":1}
I20260812 00:06:59.180883 20529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.166532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.7.85.253:43829)
0812 00:06:59.166572 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.166574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.173556 (+ 6982us) server_negotiation.cc:407] Connection header received
0812 00:06:59.173590 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.173591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.173605 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.173673 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.174633 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.175191 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.176206 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.176675 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.176767 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.177913 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.177914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.177914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.177921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.179563 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.179929 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.179930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.179931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.179984 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.180213 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.180214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.180215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.180362 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.180427 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.180581 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.180736 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":22}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.254@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
I20260812 00:06:59.186479 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.177012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.7.85.253:43829)
0812 00:06:59.177058 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.177060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.177569 (+ 509us) server_negotiation.cc:407] Connection header received
0812 00:06:59.177713 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.177714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.177726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.177761 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.178483 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.179212 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.180345 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.180679 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.180744 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.183480 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.183482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.183482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.183489 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.185039 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.185636 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.185637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.185638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.185700 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.185944 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.185945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.185945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.186060 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.186113 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.186336 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.186411 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:06:59.186542 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.176953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55190)
0812 00:06:59.177450 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.177504 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.177613 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.177842 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.177845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.178294 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.178367 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.178370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.179315 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.179316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.180263 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.180266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.180801 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.180802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.181378 (+ 576us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.181894 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.181900 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.183370 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.185133 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.185141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.185143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.185458 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.185781 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.185782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.185782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.185860 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.186124 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.186132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.186196 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.186280 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.186415 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":423,"threads_started":1}
I20260812 00:06:59.187582 19489 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.85.253:43829: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "13529a78b39e49808ed2fc0c855410f7"
member_type: NON_VOTER
last_known_addr {
host: "127.7.85.253"
port: 43829
}
attrs {
promote: true
}
}
dest_uuid: "c79758a37f514ac292fd19a499799159"
cas_config_opid_index: -1
I20260812 00:06:59.187948 19489 consensus_queue.cc:237] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } }
I20260812 00:06:59.188516 20574 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.188851 20471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:06:59.189692 20574 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.85.253:43829
I20260812 00:06:59.189872 20471 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Refusing update from remote peer c79758a37f514ac292fd19a499799159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 00:06:59.190037 20397 master.cc:537] Master@127.7.85.253:43829 shutting down...
I20260812 00:06:59.189987 20575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.190073 20575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:59.190721 20531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:59.190820 20531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } }
I20260812 00:06:59.190871 20575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:59.191020 20531 leader_election.cc:288] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135)
I20260812 00:06:59.191468 19516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13529a78b39e49808ed2fc0c855410f7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79758a37f514ac292fd19a499799159" is_pre_election: true
I20260812 00:06:59.191816 20436 leader_election.cc:302] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 13529a78b39e49808ed2fc0c855410f7; no voters: c79758a37f514ac292fd19a499799159
W20260812 00:06:59.192087 19465 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 00:06:59.192467 20531 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 00:06:59.192680 19465 consensus_peers.cc:487] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 -> Peer 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253:43829): Couldn't send request to peer 13529a78b39e49808ed2fc0c855410f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 00:06:59.193816 20397 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:06:59.193892 20397 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:06:59.193923 20397 tablet_replica.cc:330] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: stopping tablet replica
I20260812 00:06:59.207101 20397 master.cc:552] Master@127.7.85.253:43829 shutdown complete.
I20260812 00:06:59.207157 20397 master_runner.cc:305] Clearing existing system tablet
I20260812 00:06:59.207306 20397 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.207360 20397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.208024 20397 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260812 00:06:59.208572 20581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.208693 20397 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.208760 20397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.208782 20397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
uuid: "13529a78b39e49808ed2fc0c855410f7"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "8b455fb45b9d70342ad8c958ec690fef"
server_key_iv: "cbab7d33918087405bb4f6535a6b0cae"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.209704 20397 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:59.211325 20397 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 00:06:59.211395 20397 log.cc:1196] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 00:06:59.211637 20397 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Deleting consensus metadata
I20260812 00:06:59.211786 20397 master_runner.cc:315] Copying system tablet from 127.7.85.254:43135
I20260812 00:06:59.212464 20397 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: tablet copy: Beginning tablet copy session from remote peer at address 127.7.85.254:43135
I20260812 00:06:59.219609 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.212956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56952 (local address 127.7.85.254:43135)
0812 00:06:59.213044 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.213047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.213117 (+ 70us) server_negotiation.cc:407] Connection header received
0812 00:06:59.213205 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.213206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.213219 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.213276 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.213572 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.214377 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.215144 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.215457 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.215532 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.216548 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.216550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.216550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.216557 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.218416 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.218779 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.218781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.218782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.218847 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.219055 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.219056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.219056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.219174 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.219252 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.219429 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.219506 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 00:06:59.219622 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.212843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:56952)
0812 00:06:59.213034 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.213046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.213112 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.213284 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.213285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.213442 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.213500 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.213501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.214476 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.214477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.215055 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.215056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.215540 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.215541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.215596 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.216128 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.216134 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.216456 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.218514 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.218515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.218516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.218702 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.218919 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.218920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.218920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.218983 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.219244 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.219245 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.219295 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.219388 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.219508 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260812 00:06:59.220014 19531 tablet_copy_service.cc:140] P c79758a37f514ac292fd19a499799159: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 13529a78b39e49808ed2fc0c855410f7 ({username='oryx', principal='oryx/127.7.85.253@KRBTEST.COM'} at 127.0.0.1:56952)
I20260812 00:06:59.220113 19531 tablet_copy_service.cc:161] P c79758a37f514ac292fd19a499799159: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 13529a78b39e49808ed2fc0c855410f7 at {username='oryx', principal='oryx/127.7.85.253@KRBTEST.COM'} at 127.0.0.1:56952: session id = 13529a78b39e49808ed2fc0c855410f7-00000000000000000000000000000000
I20260812 00:06:59.220877 19531 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 00:06:59.221544 20397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 00:06:59.222628 20397 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: tablet copy: Starting download of 0 data blocks...
I20260812 00:06:59.222790 20397 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: tablet copy: Starting download of 1 WAL segments...
I20260812 00:06:59.224215 20397 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 00:06:59.225224 19531 tablet_copy_service.cc:342] P c79758a37f514ac292fd19a499799159: Request end of tablet copy session 13529a78b39e49808ed2fc0c855410f7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.85.253@KRBTEST.COM'} at 127.0.0.1:56952
I20260812 00:06:59.225301 19531 tablet_copy_service.cc:433] P c79758a37f514ac292fd19a499799159: ending tablet copy session 13529a78b39e49808ed2fc0c855410f7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 13529a78b39e49808ed2fc0c855410f7
I20260812 00:06:59.228506 20397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:59.229911 20591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.229940 20594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.230300 20397 server_base.cc:1034] running on GCE node
W20260812 00:06:59.230268 20598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.230496 20397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:59.230644 20397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:59.231772 20397 hybrid_clock.cc:648] HybridClock initialized: now 1786493219231776 us; error 38 us; skew 500 ppm
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:59.233390 20397 init.cc:377] Logged in from keytab as oryx/127.7.85.253@KRBTEST.COM (short username oryx)
I20260812 00:06:59.233713 20397 webserver.cc:460] Webserver started at http://127.7.85.253:41929/ using document root <none> and password file <none>
I20260812 00:06:59.233840 20397 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.233868 20397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.234539 20397 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 00:06:59.235374 20606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.235499 20397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 00:06:59.235549 20397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
uuid: "13529a78b39e49808ed2fc0c855410f7"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "8b455fb45b9d70342ad8c958ec690fef"
server_key_iv: "cbab7d33918087405bb4f6535a6b0cae"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.235612 20397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.273515 20397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:59.275404 20397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:59.278975 20397 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.253:43829
I20260812 00:06:59.279006 20720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.253:43829 every 8 connection(s)
I20260812 00:06:59.279323 20397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
I20260812 00:06:59.280181 20724 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:06:59.280661 20724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Bootstrap starting.
I20260812 00:06:59.283350 20724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:06:59.283596 20724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Bootstrap complete.
I20260812 00:06:59.283833 20724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } }
I20260812 00:06:59.283905 20724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 13529a78b39e49808ed2fc0c855410f7, State: Initialized, Role: LEARNER
I20260812 00:06:59.283934 20724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } }
I20260812 00:06:59.284125 20724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:59.284126 20732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.284243 20732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:06:59.285815 20755 catalog_manager.cc:1259] Loaded cluster ID: bca890f5cf0f40b0a221b08e88d5662f
I20260812 00:06:59.285931 20755 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: loading cluster ID for follower catalog manager: success
I20260812 00:06:59.287127 20755 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: acquiring CA information for follower catalog manager: success
I20260812 00:06:59.287448 20755 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:06:59.309450 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.300505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.7.85.253:43829)
0812 00:06:59.300681 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.300683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.300690 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:06:59.300719 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.300719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.300730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.300813 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.301477 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.302598 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.303746 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.304084 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.304169 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.305945 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.305946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.305946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.305955 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.307652 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.308393 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.308394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.308395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.308465 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.308775 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.308775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.308776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.308916 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.309017 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.309241 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.309335 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.7.85.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.252.keytab.
Entry for principal oryx/127.7.85.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.252.keytab.
I20260812 00:06:59.345770 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:35855
--webserver_interface=127.7.85.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.252.keytab
--principal=oryx/127.7.85.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:06:59.435377 20767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:59.435727 20767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:59.435779 20767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:59.437000 20767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:06:59.437063 20767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:59.437089 20767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:59.437120 20767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:06:59.437192 20767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:06:59.438737 20767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.252.keytab
--principal=oryx/127.7.85.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.85.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:59.439021 20767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:59.439263 20767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:59.444563 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.444633 20767 server_base.cc:1034] running on GCE node
W20260812 00:06:59.444569 20787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.444569 20781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.445097 20767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:59.445472 20767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:59.446651 20767 hybrid_clock.cc:648] HybridClock initialized: now 1786493219446638 us; error 47 us; skew 500 ppm
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:59.449724 20767 init.cc:377] Logged in from keytab as oryx/127.7.85.252@KRBTEST.COM (short username oryx)
I20260812 00:06:59.450222 20767 webserver.cc:460] Webserver started at http://127.7.85.252:46777/ using document root <none> and password file <none>
I20260812 00:06:59.450496 20767 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.450568 20767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.450724 20767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:06:59.451884 20767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/instance:
uuid: "9da968a3e1db4b16b7270f7fa114f67f"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "1c0934b147b4658da2fcdda05ec56234"
server_key_iv: "0d0fbea24d7b8cb00d8783d8f99197f5"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.452260 20767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal/instance:
uuid: "9da968a3e1db4b16b7270f7fa114f67f"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "1c0934b147b4658da2fcdda05ec56234"
server_key_iv: "0d0fbea24d7b8cb00d8783d8f99197f5"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.453796 20767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:06:59.455083 20802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.455371 20767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:06:59.455480 20767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
uuid: "9da968a3e1db4b16b7270f7fa114f67f"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "1c0934b147b4658da2fcdda05ec56234"
server_key_iv: "0d0fbea24d7b8cb00d8783d8f99197f5"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.455541 20767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.485684 20767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:59.489114 20767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:59.489354 20767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:59.493726 20767 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.252:35855
I20260812 00:06:59.493748 20919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.252:35855 every 8 connection(s)
I20260812 00:06:59.494170 20767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
I20260812 00:06:59.495419 20922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:06:59.497431 20922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:06:59.502734 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 20767
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.252@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:59.508235 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.498418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.7.85.254:43135)
0812 00:06:59.498520 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.498524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.498715 (+ 191us) server_negotiation.cc:407] Connection header received
0812 00:06:59.499242 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.499243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.499258 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.499323 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.500000 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.500516 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.501611 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.501821 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.501902 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.504770 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.504772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.504772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.504790 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.506540 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.507046 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.507048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.507048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.507119 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.507472 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.507474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.507474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.507641 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.507740 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.508017 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.508120 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:06:59.508248 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.498156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:56956)
0812 00:06:59.498581 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.498624 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.499125 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.499396 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.499400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.499680 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.499897 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.499901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.500621 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.500622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.501520 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.501523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.501913 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.501914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.502041 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.502586 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.502595 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.504641 (+ 2046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.506621 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.506625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.506627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.506952 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.507198 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.507199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.507199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.507380 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.507734 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.507740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.507863 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.507961 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.508079 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260812 00:06:59.508920 20922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.85.252@KRBTEST.COM
I20260812 00:06:59.513060 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.503392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.7.85.252:35855)
0812 00:06:59.503568 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.503573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.503593 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:06:59.503615 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.503617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.503639 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.503743 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.504500 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.505341 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.506395 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.506802 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.506893 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.509779 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.509786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.509787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.509794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.511667 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.512121 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.512124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.512124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.512219 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.512459 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.512460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.512461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.512654 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.512785 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.512927 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.512986 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.252@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
I20260812 00:06:59.516745 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.509223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55224)
0812 00:06:59.509323 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.509336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.509382 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.509544 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.509545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.509681 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.509723 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.509724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.510365 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.510367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.511209 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.511211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.511593 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.511594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.511676 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.512110 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.512115 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.513650 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.515479 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.515480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.515481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.515656 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.515977 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.515978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.515978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.516044 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.516410 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.516411 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.516462 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.516542 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.516648 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 00:06:59.516837 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.509287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.7.85.253:43829)
0812 00:06:59.509328 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.509330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.509368 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:06:59.509464 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.509465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.509479 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.509533 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.509823 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.510287 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.511306 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.511512 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.511573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.513751 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.513753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.513753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.513761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.515369 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.515833 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.515835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.515835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.515903 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.516125 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.516126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.516126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.516251 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.516430 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.516582 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.516710 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:06:59.517360 20922 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.252@KRBTEST.COM for oryx/127.7.85.252@KRBTEST.COM
I20260812 00:06:59.523803 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.517638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.0.0.1:39028)
0812 00:06:59.517694 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.517705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.517745 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.517862 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.517863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.517977 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.518038 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.518040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.518734 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.518736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.519202 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.519203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.519609 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.519610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.519666 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.520037 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.520041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.521278 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.522729 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.522730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.522730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.522901 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.523139 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.523141 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.523141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.523209 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.523503 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.523504 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.523553 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.523637 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.523738 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 00:06:59.523869 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.517698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.7.85.252:35855)
0812 00:06:59.517733 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.517734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.517741 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:06:59.517806 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.517807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.517816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.517860 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.518127 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.518622 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.519317 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.519519 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.519581 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.521377 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.521379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.521379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.521385 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.522620 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.522984 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.522985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.522985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.523051 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.523297 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.523298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.523298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.523416 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.523478 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.523667 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.523793 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":18}
I20260812 00:06:59.525568 20922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Bootstrap starting.
I20260812 00:06:59.526456 20922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Neither blocks nor log segments found. Creating new log.
I20260812 00:06:59.526746 20922 log.cc:824] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Log is configured to *not* fsync() on all Append() calls
I20260812 00:06:59.527653 20922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: No bootstrap required, opened a new log
I20260812 00:06:59.529278 20922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } }
I20260812 00:06:59.529425 20922 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:06:59.529474 20922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da968a3e1db4b16b7270f7fa114f67f, State: Initialized, Role: FOLLOWER
I20260812 00:06:59.529623 20922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } }
I20260812 00:06:59.529963 20922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:59.529958 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } } }
I20260812 00:06:59.530084 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:06:59.530807 20931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:06:59.530900 20931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } }
I20260812 00:06:59.531435 19516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9da968a3e1db4b16b7270f7fa114f67f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79758a37f514ac292fd19a499799159" is_pre_election: true
I20260812 00:06:59.531438 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9da968a3e1db4b16b7270f7fa114f67f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13529a78b39e49808ed2fc0c855410f7" is_pre_election: true
I20260812 00:06:59.531559 20669 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Handling vote request from an unknown peer 9da968a3e1db4b16b7270f7fa114f67f
I20260812 00:06:59.531615 20669 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 9da968a3e1db4b16b7270f7fa114f67f for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 00:06:59.531394 20931 leader_election.cc:288] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253:43829)
I20260812 00:06:59.531839 20806 leader_election.cc:302] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9da968a3e1db4b16b7270f7fa114f67f; no voters: 13529a78b39e49808ed2fc0c855410f7, c79758a37f514ac292fd19a499799159
I20260812 00:06:59.531948 20931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 0 FOLLOWER]: Advancing to term 1
W20260812 00:06:59.532258 20952 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:06:59.532305 20952 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:06:59.532891 20931 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 00:06:59.535385 20767 master_runner.cc:428] Detected that this master 9da968a3e1db4b16b7270f7fa114f67f is joining an existing cluster
I20260812 00:06:59.535454 20767 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.85.252:35855
I20260812 00:06:59.545128 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.537595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.7.85.252:35855)
0812 00:06:59.537652 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.537654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.537749 (+ 95us) server_negotiation.cc:407] Connection header received
0812 00:06:59.537883 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.537884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.537895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.537923 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.538488 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.539198 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540263 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.540550 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540592 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.542500 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.542501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.542501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.542508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.543941 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.544366 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.544368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.544368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.544428 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.544647 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.544648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.544648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.544756 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.544814 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.544993 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.545062 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 00:06:59.545295 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.537310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.0.0.1:39042)
0812 00:06:59.537676 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.537695 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.537773 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.537991 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.537991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.538292 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.538389 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.538392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.539309 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.539309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.540168 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.540171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.540674 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.540741 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.541915 (+ 1174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.541920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.542402 (+ 482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.544028 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.544029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.544029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.544260 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.544507 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.544508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.544508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.544582 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.544821 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.544822 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.544894 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.544979 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.545146 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":61,"thread_start_us":136,"threads_started":1}
I20260812 00:06:59.557937 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.537529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.7.85.254:43135)
0812 00:06:59.537624 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.537627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.537634 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:06:59.537660 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.537661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.537679 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.537730 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.538313 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.539117 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540398 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.540699 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540742 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.554772 (+ 14030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.554775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.554775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.554786 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.556571 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.556957 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.556959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.556959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.557026 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.557282 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.557283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.557284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.557449 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.557520 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.557734 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.557812 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:06:59.557976 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.536527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:56960)
0812 00:06:59.537222 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.537233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.537310 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.537782 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.537783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.538161 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.538220 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.538221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.539221 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.539222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.540294 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.540297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.540824 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.540899 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.554000 (+ 13101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.554006 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.554635 (+ 629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.556682 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.556683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.556684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.556876 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.557101 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.557102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.557102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.557227 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.557549 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.557551 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.557602 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.557688 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.557816 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":81,"thread_start_us":587,"threads_started":1}
I20260812 00:06:59.559777 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.536572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.7.85.253:43829)
0812 00:06:59.536617 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.536619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.537574 (+ 955us) server_negotiation.cc:407] Connection header received
0812 00:06:59.537704 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.537705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.537717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.537747 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.538312 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.539116 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540398 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.540708 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540744 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.555103 (+ 14359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.555104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.555105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.555115 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.557287 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.557660 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.557661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.557661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.557728 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.558012 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.558012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.558013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.558144 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.558184 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.558395 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.559637 (+ 1242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:06:59.560135 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.537015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55226)
0812 00:06:59.537504 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.537511 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.537575 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.537818 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.537818 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.538131 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.538214 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.538217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.539221 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.539222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.540294 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.540297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.540826 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.540827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.540889 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.553957 (+ 13068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.553966 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.554635 (+ 669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.557374 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.557374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.557375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.557581 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.557822 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.557823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.557823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.557936 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.558231 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.558231 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.558287 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.559873 (+ 1586us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.559989 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":102,"thread_start_us":92,"threads_started":1}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493218, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.254@KRBTEST.COM for oryx/127.7.85.252@KRBTEST.COM
I20260812 00:06:59.565152 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 19610
I20260812 00:06:59.575397 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.560722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.0.0.1:39048)
0812 00:06:59.560785 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.560797 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.560850 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.561010 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.561010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.561135 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.561220 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.561222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.562192 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.562193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.562738 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.562740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.563257 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.563259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.563326 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.563808 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.563813 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.565283 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.574108 (+ 8825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.574110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.574110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.574456 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.574692 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.574693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.574693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.574755 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.575027 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.575028 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.575074 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.575128 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.575250 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 00:06:59.575811 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.560798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.7.85.252:35855)
0812 00:06:59.560846 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.560848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.560859 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:06:59.560913 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.560914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.560927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.561009 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.561295 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.562066 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.562850 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.563151 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.563224 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.570288 (+ 7064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.570290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.570291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.570300 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.573994 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.574550 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.574551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.574552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.574615 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.574810 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.574811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.574811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.574952 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.575004 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.575595 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.575688 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:06:59.580704 19489 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.85.252:35855: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f"
member_type: NON_VOTER
last_known_addr {
host: "127.7.85.252"
port: 35855
}
attrs {
promote: true
}
}
dest_uuid: "c79758a37f514ac292fd19a499799159"
cas_config_opid_index: 5
I20260812 00:06:59.580976 19489 consensus_queue.cc:237] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } }
I20260812 00:06:59.582957 20575 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.584048 20575 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.85.252:35855
I20260812 00:06:59.584127 20575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.584206 20575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:59.584918 20861 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Refusing update from remote peer c79758a37f514ac292fd19a499799159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 00:06:59.585153 20574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:06:59.586634 20861 raft_consensus.cc:858] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } }
W20260812 00:06:59.586695 20861 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } }
I20260812 00:06:59.588886 20861 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.589445 20767 master.cc:537] Master@127.7.85.252:35855 shutting down...
I20260812 00:06:59.600210 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c79758a37f514ac292fd19a499799159. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.600318 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 00:06:59.600354 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.600397 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 00:06:59.600423 20931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } }
I20260812 00:06:59.600466 20931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 00:06:59.608825 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.608975 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
W20260812 00:06:59.610438 19467 consensus_peers.cc:487] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 -> Peer 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252:35855): Couldn't send request to peer 9da968a3e1db4b16b7270f7fa114f67f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 00:06:59.610553 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.584508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.7.85.253:43829)
0812 00:06:59.597969 (+ 13461us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.597972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.606652 (+ 8680us) server_negotiation.cc:407] Connection header received
0812 00:06:59.606795 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.606797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.606824 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.606870 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:06:59.607095 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.607878 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.609475 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.610066 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.610115 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.610191 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.610434 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.610479 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13426}
I20260812 00:06:59.611135 20767 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 00:06:59.611238 20767 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 00:06:59.611290 20767 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: stopping tablet replica
I20260812 00:06:59.611253 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.584391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55240)
0812 00:06:59.606547 (+ 22156us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.606565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.606702 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.606962 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.606963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.606965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:06:59.607021 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.607024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.607986 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.607987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.609365 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.609369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.610257 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.610258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.610325 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.610349 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.610884 (+ 535us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.610936 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22123}
I20260812 00:06:59.611776 20669 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Refusing update from remote peer c79758a37f514ac292fd19a499799159: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 00:06:59.612020 20575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 00:06:59.612625 20669 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.613075 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:40603
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:43135
--webserver_port=41955
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:06:59.613879 20732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c79758a37f514ac292fd19a499799159. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.613963 20732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:06:59.613996 20732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: NON_VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: true } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.614035 20732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:06:59.614030 20575 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: attempting to promote NON_VOTER 13529a78b39e49808ed2fc0c855410f7 to VOTER
I20260812 00:06:59.614234 20575 consensus_queue.cc:237] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } }
W20260812 00:06:59.615075 19467 consensus_peers.cc:487] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 -> Peer 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252:35855): Couldn't send request to peer 9da968a3e1db4b16b7270f7fa114f67f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 00:06:59.615386 20669 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 LEARNER]: Refusing update from remote peer c79758a37f514ac292fd19a499799159: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 00:06:59.615588 20971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 00:06:59.616303 20971 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.617529 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.617630 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:59.617703 20971 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.617795 20971 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:06:59.618101 20669 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.619004 20997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.619127 20997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:06:59.631053 20767 master.cc:552] Master@127.7.85.252:35855 shutdown complete.
I20260812 00:06:59.631115 20767 master_runner.cc:305] Clearing existing system tablet
I20260812 00:06:59.631260 20767 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.631311 20767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.632071 20767 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:59.632656 21006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.632824 20767 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.632889 20767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.632915 20767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
uuid: "9da968a3e1db4b16b7270f7fa114f67f"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "1c0934b147b4658da2fcdda05ec56234"
server_key_iv: "0d0fbea24d7b8cb00d8783d8f99197f5"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.633733 20767 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:06:59.635421 20767 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 00:06:59.635501 20767 log.cc:1196] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 00:06:59.635777 20767 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Deleting consensus metadata
I20260812 00:06:59.636039 20767 master_runner.cc:315] Copying system tablet from 127.7.85.254:43135
I20260812 00:06:59.636696 20767 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: tablet copy: Beginning tablet copy session from remote peer at address 127.7.85.254:43135
I20260812 00:06:59.644336 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.637004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:56974)
0812 00:06:59.637161 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.637204 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.637416 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.637599 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.637600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.637816 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.637876 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.637878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.638861 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.638862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.639648 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.639651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.640122 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.640123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.640202 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.640750 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.640756 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.641220 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.643126 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.643127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.643128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.643373 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.643604 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.643605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.643605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.643679 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.643980 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.643981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.644030 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.644106 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.644231 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260812 00:06:59.644349 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.637080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.7.85.254:43135)
0812 00:06:59.637149 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.637152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.637427 (+ 275us) server_negotiation.cc:407] Connection header received
0812 00:06:59.637501 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.637503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.637516 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.637577 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.637969 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.638768 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.639747 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.640035 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.640108 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.641317 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.641318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.641319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.641328 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.643029 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.643458 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.643459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.643459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.643525 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.643752 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.643753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.643753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.643898 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.643971 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.644153 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.644244 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:06:59.645128 19531 tablet_copy_service.cc:140] P c79758a37f514ac292fd19a499799159: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9da968a3e1db4b16b7270f7fa114f67f ({username='oryx', principal='oryx/127.7.85.252@KRBTEST.COM'} at 127.0.0.1:56974)
I20260812 00:06:59.645231 19531 tablet_copy_service.cc:161] P c79758a37f514ac292fd19a499799159: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9da968a3e1db4b16b7270f7fa114f67f at {username='oryx', principal='oryx/127.7.85.252@KRBTEST.COM'} at 127.0.0.1:56974: session id = 9da968a3e1db4b16b7270f7fa114f67f-00000000000000000000000000000000
I20260812 00:06:59.645815 19531 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 00:06:59.646324 20767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 00:06:59.647719 20767 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: tablet copy: Starting download of 0 data blocks...
I20260812 00:06:59.647867 20767 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: tablet copy: Starting download of 1 WAL segments...
I20260812 00:06:59.648841 20767 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 00:06:59.649854 19531 tablet_copy_service.cc:342] P c79758a37f514ac292fd19a499799159: Request end of tablet copy session 9da968a3e1db4b16b7270f7fa114f67f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.85.252@KRBTEST.COM'} at 127.0.0.1:56974
I20260812 00:06:59.649920 19531 tablet_copy_service.cc:433] P c79758a37f514ac292fd19a499799159: ending tablet copy session 9da968a3e1db4b16b7270f7fa114f67f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9da968a3e1db4b16b7270f7fa114f67f
I20260812 00:06:59.652805 20767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:59.654151 21017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.654264 21025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.654523 20767 server_base.cc:1034] running on GCE node
W20260812 00:06:59.654534 21021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.654875 20767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:59.655025 20767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:59.656153 20767 hybrid_clock.cc:648] HybridClock initialized: now 1786493219656162 us; error 45 us; skew 500 ppm
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:59.657727 20767 init.cc:377] Logged in from keytab as oryx/127.7.85.252@KRBTEST.COM (short username oryx)
I20260812 00:06:59.658003 20767 webserver.cc:460] Webserver started at http://127.7.85.252:37531/ using document root <none> and password file <none>
I20260812 00:06:59.658128 20767 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.658164 20767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.658959 20767 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:59.659566 21038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.659734 20767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.659788 20767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
uuid: "9da968a3e1db4b16b7270f7fa114f67f"
format_stamp: "Formatted at 2026-08-12 00:06:59 on dist-test-slave-z8x9"
server_key: "1c0934b147b4658da2fcdda05ec56234"
server_key_iv: "0d0fbea24d7b8cb00d8783d8f99197f5"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.659847 20767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.674369 20767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:59.677038 20767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:59.680578 20767 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.252:35855
I20260812 00:06:59.680795 21149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.252:35855 every 8 connection(s)
I20260812 00:06:59.681391 20767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
I20260812 00:06:59.682392 21152 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:06:59.682960 21152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Bootstrap starting.
I20260812 00:06:59.686237 21152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:06:59.686520 21152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: Bootstrap complete.
I20260812 00:06:59.686803 21152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } }
I20260812 00:06:59.686909 21152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9da968a3e1db4b16b7270f7fa114f67f, State: Initialized, Role: LEARNER
I20260812 00:06:59.686955 21152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } }
I20260812 00:06:59.687162 21152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:06:59.687191 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:06:59.687417 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: LEARNER
I20260812 00:06:59.689682 21190 catalog_manager.cc:1259] Loaded cluster ID: bca890f5cf0f40b0a221b08e88d5662f
I20260812 00:06:59.689740 21190 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: loading cluster ID for follower catalog manager: success
I20260812 00:06:59.690838 21190 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: acquiring CA information for follower catalog manager: success
I20260812 00:06:59.691131 21190 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 00:06:59.720523 20998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:59.720767 20998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:59.720793 20998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:59.722111 20998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:59.722160 20998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:59.722189 20998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:06:59.723781 20998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:40603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=41955
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:59.724035 20998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:59.724313 20998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:59.730436 21203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.730473 21208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.730588 20998 server_base.cc:1034] running on GCE node
W20260812 00:06:59.730772 21202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.730984 20998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:59.731244 20998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:59.732429 20998 hybrid_clock.cc:648] HybridClock initialized: now 1786493219732404 us; error 39 us; skew 500 ppm
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:59.735978 20998 init.cc:377] Logged in from keytab as oryx/127.7.85.193@KRBTEST.COM (short username oryx)
I20260812 00:06:59.736516 20998 webserver.cc:460] Webserver started at http://127.7.85.193:41955/ using document root <none> and password file <none>
I20260812 00:06:59.736765 20998 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.736860 20998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.738651 20998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:59.739431 21218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.739701 20998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:59.739811 20998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "330181aaf8134023a96c5e42e73e08b0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "230c93c10915935b4f8a124562544f14"
server_key_iv: "76b77ebe3ee0f2ca571e83ea12a15d58"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.740219 20998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.761946 20112 heartbeater.cc:503] Master 127.7.85.254:43135 was elected leader, sending a full tablet report...
I20260812 00:06:59.774140 20998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:59.778239 20998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:59.778503 20998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:59.779081 20998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:59.779773 20998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:59.779860 20998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.779909 20998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:59.779938 20998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.790032 20998 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:40603
I20260812 00:06:59.790236 21437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:40603 every 8 connection(s)
I20260812 00:06:59.790561 20998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:06:59.792090 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 20998
I20260812 00:06:59.792275 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 19878
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.252@KRBTEST.COM
I20260812 00:06:59.815239 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.791722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:38771 (local address 127.7.85.252:35855)
0812 00:06:59.791914 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.791916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.791926 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:06:59.793053 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.793055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.793071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.793114 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.794217 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.795098 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.796532 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.796768 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.796857 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.799533 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.799534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.799535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.799544 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.801164 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.814061 (+ 12897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.814065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.814065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.814155 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.814432 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.814433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.814433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.814581 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.814642 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.814987 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.815099 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":62,"threads_started":1}
I20260812 00:06:59.816515 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.791438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.7.85.193:38771)
0812 00:06:59.791641 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.791686 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.792929 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.793426 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.793431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.793826 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.794099 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.794103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.795207 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.795208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.796426 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.796429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.796865 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.796866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.797013 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.797779 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.797789 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.799410 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.813449 (+ 14039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.813466 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.813470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.813942 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.814223 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.814223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.814224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.814368 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.814672 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.814678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.814853 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.815909 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.816196 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
I20260812 00:06:59.817485 21443 heartbeater.cc:347] Connected to a master server at 127.7.85.252:35855
I20260812 00:06:59.817658 21443 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:59.817914 21443 heartbeater.cc:511] Master 127.7.85.252:35855 requested a full tablet report, sending...
I20260812 00:06:59.819139 21082 ts_manager.cc:194] Registered new tserver with Master: 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603)
I20260812 00:06:59.822185 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.791436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:34391 (local address 127.7.85.253:43829)
0812 00:06:59.791505 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.791507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.791769 (+ 262us) server_negotiation.cc:407] Connection header received
0812 00:06:59.802209 (+ 10440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.802212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.802229 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.802272 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.812438 (+ 10166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.813382 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.814712 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.815112 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.815189 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.817790 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.817791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.817792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.817800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.820018 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.820454 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.820456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.820456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.820527 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.820747 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.820749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.820749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.820882 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.820967 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.821998 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.822086 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:06:59.822219 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.791439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.7.85.193:34391)
0812 00:06:59.791638 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.791686 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.802048 (+ 10362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.811804 (+ 9756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.811807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.812147 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.812267 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.812272 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.813541 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.813542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.814587 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.814591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.815261 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.815262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.815355 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.815928 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.815933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.817689 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.820133 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.820134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.820134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.820361 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.820598 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.820599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.820599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.820668 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.821822 (+ 1154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.821823 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.821869 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.821950 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.822083 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"spinlock_wait_cycles":768,"thread_start_us":138,"threads_started":1}
I20260812 00:06:59.822427 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:46105
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:43135
--webserver_port=36549
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:59.822770 21444 heartbeater.cc:347] Connected to a master server at 127.7.85.253:43829
I20260812 00:06:59.822832 21444 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:59.822923 21444 heartbeater.cc:511] Master 127.7.85.253:43829 requested a full tablet report, sending...
I20260812 00:06:59.823339 20652 ts_manager.cc:194] Registered new tserver with Master: 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603)
I20260812 00:06:59.827242 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.791789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:47163)
0812 00:06:59.791978 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.791990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.816601 (+ 24611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.816856 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.816857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.817040 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.817083 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.817085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.818075 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.818076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.818998 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.819001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.819705 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.819706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.819773 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.820286 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.820292 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.823080 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.825980 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.825982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.825982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.826193 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.826473 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.826475 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.826475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.826559 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.826935 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.826936 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.826983 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.827038 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.827140 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":67,"threads_started":1}
I20260812 00:06:59.827353 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.813580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:47163 (local address 127.7.85.254:43135)
0812 00:06:59.813649 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.813651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.813660 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:06:59.816715 (+ 3055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.816718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.816736 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.816772 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.817228 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.817974 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.819119 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.819531 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.819657 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.823547 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.823549 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.823550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.823559 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.825875 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.826296 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.826297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.826298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.826375 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.826645 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.826646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.826647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.826827 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.826904 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.827106 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.827213 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:06:59.827704 21439 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43135
I20260812 00:06:59.827776 21439 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:59.827910 21439 heartbeater.cc:511] Master 127.7.85.254:43135 requested a full tablet report, sending...
I20260812 00:06:59.828229 19489 ts_manager.cc:194] Re-registered known tserver with Master: 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603)
I20260812 00:06:59.828806 19489 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:47163
W20260812 00:06:59.911319 21459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:06:59.911532 21459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:06:59.911557 21459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:06:59.912698 21459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:06:59.912738 21459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:06:59.912765 21459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:06:59.914227 21459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:46105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=36549
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:06:59.914480 21459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:06:59.914754 21459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:06:59.920499 21475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.920512 21472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:06:59.920512 21468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:06:59.920958 21459 server_base.cc:1034] running on GCE node
I20260812 00:06:59.921145 21459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:06:59.921475 21459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:06:59.922641 21459 hybrid_clock.cc:648] HybridClock initialized: now 1786493219922621 us; error 34 us; skew 500 ppm
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:06:59.925573 21459 init.cc:377] Logged in from keytab as oryx/127.7.85.194@KRBTEST.COM (short username oryx)
I20260812 00:06:59.926095 21459 webserver.cc:460] Webserver started at http://127.7.85.194:36549/ using document root <none> and password file <none>
I20260812 00:06:59.926373 21459 fs_manager.cc:359] Metadata directory not provided
I20260812 00:06:59.926477 21459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:06:59.928052 21459 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:59.928879 21485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.929111 21459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:06:59.929260 21459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "b72df70ea65b44d9bbcef5d76e3983db"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "9fe798a5de4d86d1c6f5347d35bc8c11"
server_key_iv: "dd19cd716b6433962800e6bb8a7d1819"
server_key_version: "encryptionkey@0"
I20260812 00:06:59.929646 21459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:06:59.943172 21459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:06:59.945696 21459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:06:59.945824 21459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:06:59.946122 21459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:06:59.946452 21459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:06:59.946492 21459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.946516 21459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:06:59.946563 21459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:06:59.952872 21459 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:46105
I20260812 00:06:59.952947 21675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:46105 every 8 connection(s)
I20260812 00:06:59.953352 21459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:06:59.956532 20390 heartbeater.cc:503] Master 127.7.85.254:43135 was elected leader, sending a full tablet report...
I20260812 00:06:59.960242 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 21459
I20260812 00:06:59.960438 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 20137
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.252@KRBTEST.COM
I20260812 00:06:59.979746 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.954314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:35367 (local address 127.7.85.252:35855)
0812 00:06:59.954366 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.954368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.954760 (+ 392us) server_negotiation.cc:407] Connection header received
0812 00:06:59.955180 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.955181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.955196 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.955249 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.956232 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.956975 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.958585 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.958911 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.958993 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.975191 (+ 16198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.975193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.975194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.975203 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.977995 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.978569 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.978571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.978571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.978639 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.978917 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.978917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.978918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.979064 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.979106 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.979484 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.979592 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:06:59.980322 21693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.954235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.7.85.194:35367)
0812 00:06:59.954701 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.954711 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.955091 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.955389 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.955395 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.955840 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.956100 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.956107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.957376 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.957377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.958461 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.958464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.959150 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.959151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.959201 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.973026 (+ 13825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.973038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.975035 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.978112 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.978125 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.978128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.978485 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.978710 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.978711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.978711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.978856 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.979149 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.979155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.979343 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.979845 (+ 502us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.980048 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":12,"thread_start_us":94,"threads_started":1}
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
I20260812 00:06:59.986182 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.954229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:35133 (local address 127.7.85.253:43829)
0812 00:06:59.954303 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.954306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.954568 (+ 262us) server_negotiation.cc:407] Connection header received
0812 00:06:59.955301 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.955302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.955333 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.955383 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.956202 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.956979 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.958585 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.958911 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.958993 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.981890 (+ 22897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.981893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.981893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.981903 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.984892 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.985399 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.985401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.985401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.985454 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.985623 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.985624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.985624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.985770 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.985815 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.985986 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.986073 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:06:59.986601 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.954234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.7.85.194:35133)
0812 00:06:59.954419 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.954462 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.955077 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.955406 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.955407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.955835 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.956098 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.956103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.957090 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.957091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.958461 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.958464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.959012 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.959013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.959183 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.959947 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.959959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.981769 (+ 21810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.985011 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.985013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.985013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.985323 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.985515 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.985516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.985516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.985569 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.985845 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.985846 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.985891 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.986277 (+ 386us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.986402 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":76,"thread_start_us":90,"threads_started":1}
I20260812 00:06:59.987393 21677 heartbeater.cc:347] Connected to a master server at 127.7.85.252:35855
I20260812 00:06:59.987540 21677 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:59.987766 21677 heartbeater.cc:511] Master 127.7.85.252:35855 requested a full tablet report, sending...
I20260812 00:06:59.988722 21082 ts_manager.cc:194] Registered new tserver with Master: b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194:46105)
I20260812 00:06:59.993201 21682 heartbeater.cc:347] Connected to a master server at 127.7.85.253:43829
I20260812 00:06:59.993301 21682 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:59.993410 21682 heartbeater.cc:511] Master 127.7.85.253:43829 requested a full tablet report, sending...
Aug 12 00:06:59 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493219, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:06:59.993777 20652 ts_manager.cc:194] Registered new tserver with Master: b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194:46105)
I20260812 00:06:59.997431 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.954691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:56453 (local address 127.7.85.254:43135)
0812 00:06:59.954742 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 00:06:59.954744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:06:59.955344 (+ 600us) server_negotiation.cc:407] Connection header received
0812 00:06:59.955819 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:06:59.955821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:06:59.955834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:06:59.955891 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:06:59.956209 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.956944 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.959162 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.959361 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.959555 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.994097 (+ 34542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:06:59.994099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:06:59.994100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:06:59.994110 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:06:59.995707 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.996102 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.996104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.996104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.996171 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:06:59.996396 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:06:59.996397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:06:59.996397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:06:59.996536 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:06:59.996618 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 00:06:59.997155 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 00:06:59.997317 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:06:59.997454 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:06:59.954589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:56453)
0812 00:06:59.955267 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 00:06:59.955276 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:06:59.955704 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:06:59.955908 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:06:59.955908 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:06:59.956065 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:06:59.956114 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.956116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.957090 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.957091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.959042 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:06:59.959044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:06:59.959559 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:06:59.959561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:06:59.959641 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:06:59.960155 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:06:59.960169 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:06:59.994008 (+ 33839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:06:59.995830 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.995832 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.995833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.996023 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.996248 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:06:59.996249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:06:59.996249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:06:59.996329 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:06:59.996608 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:06:59.996609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:06:59.996664 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 00:06:59.996982 (+ 318us) client_negotiation.cc:240] Negotiation successful
0812 00:06:59.997322 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":70,"threads_started":1}
I20260812 00:06:59.997787 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:36111
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:43135
--webserver_port=32969
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:39607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:06:59.997833 21676 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43135
I20260812 00:06:59.997892 21676 heartbeater.cc:464] Registering TS with master...
I20260812 00:06:59.997974 21676 heartbeater.cc:511] Master 127.7.85.254:43135 requested a full tablet report, sending...
I20260812 00:06:59.998219 19489 ts_manager.cc:194] Re-registered known tserver with Master: b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194:46105)
I20260812 00:06:59.998575 19489 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:56453
I20260812 00:07:00.039805 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.036010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39058 (local address 127.7.85.252:35855)
0812 00:07:00.036101 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.036104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.036115 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:07:00.036150 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.036152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.036179 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.036295 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:00.036694 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.037498 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.038694 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.039319 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.039410 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.039476 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.039652 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.039721 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:07:00.039948 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.035896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.0.0.1:39058)
0812 00:07:00.035961 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.035977 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.036034 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.036429 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.036432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.036509 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:00.036585 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.036588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.037631 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.037633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.038554 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.038557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.039471 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.039472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.039564 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.039569 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.039631 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.039721 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 00:07:00.040463 21104 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260812 00:07:00.040711 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c79758a37f514ac292fd19a499799159. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:07:00.040906 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: LEARNER
W20260812 00:07:00.091115 21695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:00.091361 21695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:00.091387 21695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:00.092806 21695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:00.092872 21695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:00.092904 21695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:00.094580 21695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:36111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=32969
--tserver_master_addrs=127.7.85.254:43135,127.7.85.253:43829,127.7.85.252:35855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:00.094852 21695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:00.095127 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:07:00.101528 21695 server_base.cc:1034] running on GCE node
W20260812 00:07:00.101504 21706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:00.101495 21710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:00.101509 21705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:00.102029 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:00.102330 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:00.103540 21695 hybrid_clock.cc:648] HybridClock initialized: now 1786493220103506 us; error 57 us; skew 500 ppm
Aug 12 00:07:00 dist-test-slave-z8x9 krb5kdc[19407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493220, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:00.106717 21695 init.cc:377] Logged in from keytab as oryx/127.7.85.195@KRBTEST.COM (short username oryx)
I20260812 00:07:00.107314 21695 webserver.cc:460] Webserver started at http://127.7.85.195:32969/ using document root <none> and password file <none>
I20260812 00:07:00.107585 21695 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:00.107697 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:00.109586 21695 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:07:00.110495 21725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:00.110772 21695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:00.110908 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0"
format_stamp: "Formatted at 2026-08-12 00:06:58 on dist-test-slave-z8x9"
server_key: "7c554c26985f30bacc6e2626607b804d"
server_key_iv: "59b65350250824df92161199036ba488"
server_key_version: "encryptionkey@0"
I20260812 00:07:00.111322 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:00.129714 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:00.132925 21695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:00.133147 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:00.133852 21695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:00.134234 21695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:00.134294 21695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:00.134342 21695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:00.134369 21695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:00.141661 21695 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:36111
I20260812 00:07:00.141952 21951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:36111 every 8 connection(s)
I20260812 00:07:00.142151 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:07:00.145800 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 21695
Aug 12 00:07:00 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493220, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.253@KRBTEST.COM
Aug 12 00:07:00 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493220, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
Aug 12 00:07:00 dist-test-slave-z8x9 krb5kdc[19407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493220, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.252@KRBTEST.COM
I20260812 00:07:00.158568 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.144156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:60567 (local address 127.7.85.253:43829)
0812 00:07:00.144219 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.144221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.144439 (+ 218us) server_negotiation.cc:407] Connection header received
0812 00:07:00.144790 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.144792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.144804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.144836 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.145986 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.146771 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148460 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.148780 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148816 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.154623 (+ 5807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.154624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.154625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.154633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.156485 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.157121 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.157122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.157123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.157319 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.157672 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.157673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.157673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.157831 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.157878 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.158369 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.158464 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:07:00.159344 21450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.143587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:53391 (local address 127.7.85.252:35855)
0812 00:07:00.143663 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.143665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.143896 (+ 231us) server_negotiation.cc:407] Connection header received
0812 00:07:00.144705 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.144706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.144723 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.144793 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.145975 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.146740 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148460 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.148768 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148813 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.154804 (+ 5991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.154805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.154805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.154814 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.157103 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.157999 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.158001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.158001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.158071 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.158300 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.158301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.158301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.158448 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.158495 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.158944 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.159042 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:07:00.159988 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.144069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.7.85.195:60567)
0812 00:07:00.144330 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.144340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.144643 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.144902 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.144902 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.145545 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:00.145852 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.145869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.146889 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.146890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.148352 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.148355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.148910 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.149070 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.149981 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:00.149994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:00.154510 (+ 4516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:00.156581 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:00.156594 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:00.156598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:00.156998 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:00.157416 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:00.157417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:00.157417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:00.157584 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:00.157932 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:00.157939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:00.158167 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.158987 (+ 820us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.159254 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":119,"thread_start_us":190,"threads_started":1}
I20260812 00:07:00.160292 19576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.143991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:41623 (local address 127.7.85.254:43135)
0812 00:07:00.144058 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.144060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.144413 (+ 353us) server_negotiation.cc:407] Connection header received
0812 00:07:00.144772 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.144773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.144788 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.144823 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.145987 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.146793 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148449 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.148752 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148802 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.154122 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.154124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.154124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.154134 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.158540 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.158979 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.158980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.158981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.159041 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.159263 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.159265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.159265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.159414 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.159488 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.160099 (+ 611us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.160193 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 00:07:00.160251 21959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.143499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.7.85.195:53391)
0812 00:07:00.143697 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.143743 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.144602 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.144870 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.144875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.145539 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:00.145850 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.145856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.146885 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.146885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.148352 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.148355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.148911 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.149076 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.149981 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:00.149994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:00.154701 (+ 4707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:00.157510 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:00.157512 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:00.157512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:00.157884 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:00.158155 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:00.158156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:00.158156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:00.158220 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:00.158545 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:00.158546 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:00.158596 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.158993 (+ 397us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.159645 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":878,"thread_start_us":101,"threads_started":1}
I20260812 00:07:00.160444 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.143860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:41623)
0812 00:07:00.144312 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.144323 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.144643 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.144951 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.144952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.145500 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:00.145852 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.145869 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.146889 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.146890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.148353 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.148356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.148907 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.148908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.149069 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.150071 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:00.150076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:00.153971 (+ 3895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:00.158622 (+ 4651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:00.158624 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:00.158624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:00.158892 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:00.159113 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:00.159113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:00.159114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:00.159174 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:00.159501 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:00.159502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:00.159549 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.159612 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.159742 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":435,"thread_start_us":70,"threads_started":1}
I20260812 00:07:00.160758 21956 heartbeater.cc:347] Connected to a master server at 127.7.85.253:43829
I20260812 00:07:00.160887 21956 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:00.160964 21980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.147549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56988 (local address 127.7.85.254:43135)
0812 00:07:00.148303 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.148305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.148312 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:07:00.148335 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.148335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.148348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.148415 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.149283 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.150084 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.151118 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.151455 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.151530 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.155477 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.155478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.155479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.155488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.157728 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.158117 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.158119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.158119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.158183 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.158414 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.158415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.158415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.158557 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.159321 (+ 764us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.160737 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.160846 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":45,"threads_started":1}
I20260812 00:07:00.161106 21956 heartbeater.cc:511] Master 127.7.85.253:43829 requested a full tablet report, sending...
I20260812 00:07:00.161273 21953 heartbeater.cc:347] Connected to a master server at 127.7.85.252:35855
I20260812 00:07:00.161316 21953 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:00.161377 21953 heartbeater.cc:511] Master 127.7.85.252:35855 requested a full tablet report, sending...
I20260812 00:07:00.161679 20652 ts_manager.cc:194] Registered new tserver with Master: 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:00.162509 21082 ts_manager.cc:194] Registered new tserver with Master: 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:00.163159 21952 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43135
I20260812 00:07:00.163239 21952 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:00.163316 21952 heartbeater.cc:511] Master 127.7.85.254:43135 requested a full tablet report, sending...
I20260812 00:07:00.163579 19489 ts_manager.cc:194] Re-registered known tserver with Master: 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:00.164132 19489 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:41623
I20260812 00:07:00.164789 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.147892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.7.85.252:35855)
0812 00:07:00.148106 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.148108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.151742 (+ 3634us) server_negotiation.cc:407] Connection header received
0812 00:07:00.151770 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.151771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.151796 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.151852 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.152562 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.153353 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.154451 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.154804 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.154874 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.156763 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.156764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.156765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.156773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.163313 (+ 6540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.163746 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.163748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.163748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.163814 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.164003 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.164004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.164004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.164125 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.164236 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.164607 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.164694 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":64,"threads_started":1}
I20260812 00:07:00.165954 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.147416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.7.85.253:43829)
0812 00:07:00.150313 (+ 2897us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.150316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.150434 (+ 118us) server_negotiation.cc:407] Connection header received
0812 00:07:00.150742 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.150744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.150759 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.150833 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.151580 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.152378 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.153435 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.153750 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.153825 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.162254 (+ 8429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.162257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.162257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.162264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.164442 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.164795 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.164796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.164797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.164866 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.165087 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.165088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.165089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.165276 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.165377 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.165770 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.165861 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2733,"thread_start_us":49,"threads_started":1}
I20260812 00:07:00.164425 19489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:07:00.166795 19489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:00.167639 21991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: NON_VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: true } } }
I20260812 00:07:00.167754 21991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:00.168604 21992 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: attempting to promote NON_VOTER 9da968a3e1db4b16b7270f7fa114f67f to VOTER
I20260812 00:07:00.168756 21992 consensus_queue.cc:237] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:00.169257 20669 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Refusing update from remote peer c79758a37f514ac292fd19a499799159: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 00:07:00.169294 21104 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 LEARNER]: Refusing update from remote peer c79758a37f514ac292fd19a499799159: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260812 00:07:00.169477 21991 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 00:07:00.169572 20575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 00:07:00.170682 21992 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.170811 20669 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.170810 21104 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.171792 21992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.171794 21991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.171902 21991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:00.171906 21161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.171970 21991 sys_catalog.cc:455] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.171991 21161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:07:00.172008 21991 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:00.172029 22011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c79758a37f514ac292fd19a499799159" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:00.172122 22011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:07:00.171890 21992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:00.179376 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.175320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36111 (local address 127.0.0.1:52442)
0812 00:07:00.175550 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.175561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.175604 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.176039 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.176040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.176042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:00.176094 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.176095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.177249 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.177251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.178118 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.178120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.179107 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.179109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.179192 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.179197 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.179229 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.179277 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":138,"threads_started":1}
I20260812 00:07:00.179925 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.175430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.7.85.195:36111)
0812 00:07:00.175868 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.175873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.175883 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:00.175921 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.175923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.175946 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.176042 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:00.176150 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.177085 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.178220 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.178999 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.179639 (+ 640us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.179749 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.179781 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.179831 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":91,"threads_started":1}
I20260812 00:07:00.180698 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.175308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:46105 (local address 127.0.0.1:47036)
0812 00:07:00.175689 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.175697 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.175736 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.176082 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.176083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.176085 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:00.176124 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.176125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.177561 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.177562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.178620 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.178624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.179898 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.179906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.179977 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.179983 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.180334 (+ 351us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.180398 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":21,"thread_start_us":135,"threads_started":1}
I20260812 00:07:00.180652 21832 tablet_service.cc:1467] Processing CreateTablet for tablet b2338697d94c4392ba9e7ba13453462a (DEFAULT_TABLE table=test-table [id=c7c2c3a208744dfb9874d3168cfa03a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:00.181064 21832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2338697d94c4392ba9e7ba13453462a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:00.181816 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.175704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.7.85.194:46105)
0812 00:07:00.175909 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.175913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.175922 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:00.175962 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.175965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.175982 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.176196 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:00.176252 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.177456 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.178744 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.179713 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.179798 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.179877 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.180935 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.180993 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260812 00:07:00.182154 22015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.175307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40603 (local address 127.0.0.1:54954)
0812 00:07:00.175530 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.175544 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.175604 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.175999 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.176001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.176005 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:00.176080 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.176082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.177825 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.177827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.178861 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.178872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.181820 (+ 2948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.181822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.181935 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.181943 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.181990 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.182063 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260812 00:07:00.183826 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.175561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.7.85.193:40603)
0812 00:07:00.175755 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.175761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.175773 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:07:00.175844 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.175847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.175867 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.176176 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:00.176238 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.177735 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.179094 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.180943 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.182411 (+ 1468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.182986 (+ 575us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.183167 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.183535 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
I20260812 00:07:00.184448 21573 tablet_service.cc:1467] Processing CreateTablet for tablet b2338697d94c4392ba9e7ba13453462a (DEFAULT_TABLE table=test-table [id=c7c2c3a208744dfb9874d3168cfa03a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:00.184440 21310 tablet_service.cc:1467] Processing CreateTablet for tablet b2338697d94c4392ba9e7ba13453462a (DEFAULT_TABLE table=test-table [id=c7c2c3a208744dfb9874d3168cfa03a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:00.184841 21310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2338697d94c4392ba9e7ba13453462a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:00.184854 21573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2338697d94c4392ba9e7ba13453462a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:00.188858 22030 tablet_bootstrap.cc:492] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Bootstrap starting.
I20260812 00:07:00.189049 22028 tablet_bootstrap.cc:492] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Bootstrap starting.
I20260812 00:07:00.189376 22033 tablet_bootstrap.cc:492] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Bootstrap starting.
I20260812 00:07:00.189648 22028 tablet_bootstrap.cc:654] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:00.189692 22030 tablet_bootstrap.cc:654] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:00.189898 22028 log.cc:824] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:00.190071 22030 log.cc:824] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:00.190418 22033 tablet_bootstrap.cc:654] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:00.190838 22030 tablet_bootstrap.cc:492] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: No bootstrap required, opened a new log
I20260812 00:07:00.190838 22028 tablet_bootstrap.cc:492] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: No bootstrap required, opened a new log
I20260812 00:07:00.190837 22033 log.cc:824] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:00.190940 22030 ts_tablet_manager.cc:1397] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:00.190938 22028 ts_tablet_manager.cc:1397] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:00.191663 22033 tablet_bootstrap.cc:492] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: No bootstrap required, opened a new log
I20260812 00:07:00.191740 22033 ts_tablet_manager.cc:1397] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:00.193331 22030 raft_consensus.cc:348] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.193504 22030 raft_consensus.cc:374] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:00.193550 22030 raft_consensus.cc:729] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330181aaf8134023a96c5e42e73e08b0, State: Initialized, Role: FOLLOWER
I20260812 00:07:00.193696 22030 consensus_queue.cc:260] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.194108 22028 raft_consensus.cc:348] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.194290 22028 raft_consensus.cc:374] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:00.194327 22028 raft_consensus.cc:729] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a23078d725d4b9e8dc6d6900e3ae6f0, State: Initialized, Role: FOLLOWER
I20260812 00:07:00.194231 22033 raft_consensus.cc:348] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.194370 22033 raft_consensus.cc:374] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:00.194403 22033 raft_consensus.cc:729] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b72df70ea65b44d9bbcef5d76e3983db, State: Initialized, Role: FOLLOWER
I20260812 00:07:00.194471 22028 consensus_queue.cc:260] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.194525 22033 consensus_queue.cc:260] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.194799 22033 ts_tablet_manager.cc:1428] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:07:00.194799 22028 ts_tablet_manager.cc:1428] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 00:07:00.194933 21676 heartbeater.cc:503] Master 127.7.85.254:43135 was elected leader, sending a full tablet report...
I20260812 00:07:00.194933 21952 heartbeater.cc:503] Master 127.7.85.254:43135 was elected leader, sending a full tablet report...
I20260812 00:07:00.199628 21439 heartbeater.cc:503] Master 127.7.85.254:43135 was elected leader, sending a full tablet report...
I20260812 00:07:00.200269 22030 ts_tablet_manager.cc:1428] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
W20260812 00:07:00.204344 21684 tablet.cc:2369] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:07:00.291678 21445 tablet.cc:2369] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:07:00.393747 21958 tablet.cc:2369] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:07:00.518152 22039 raft_consensus.cc:482] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:00.518368 22039 raft_consensus.cc:504] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.518839 22039 leader_election.cc:288] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603), 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:00.522667 22016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.519209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53265 (local address 127.7.85.195:36111)
0812 00:07:00.519267 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.519269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.519368 (+ 99us) server_negotiation.cc:407] Connection header received
0812 00:07:00.519450 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.519451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.519466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.519527 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:00.519874 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.520418 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.521655 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.522231 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.522316 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.522328 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.522569 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.522611 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 00:07:00.522694 22045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.519058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36111 (local address 127.7.85.194:53265)
0812 00:07:00.519266 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.519279 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.519374 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.519542 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.519543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.519546 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:00.519633 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.519635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.520512 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.520513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.521494 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.521497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.522350 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.522352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.522434 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.522439 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.522515 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.522565 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":115,"threads_started":1}
I20260812 00:07:00.523290 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.519289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53425 (local address 127.7.85.193:40603)
0812 00:07:00.519364 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.519366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.519423 (+ 57us) server_negotiation.cc:407] Connection header received
0812 00:07:00.519516 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.519517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.519532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.519593 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:00.520074 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.520628 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.522345 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.522904 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.522971 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.522979 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.523184 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.523227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:07:00.523290 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.519127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40603 (local address 127.7.85.194:53425)
0812 00:07:00.519324 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:00.519337 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:00.519424 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:00.519630 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:00.519631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:00.519634 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:00.519877 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.519880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.520810 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.520812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.522220 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:00.522223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.523014 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.523015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:00.523079 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.523083 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:00.523169 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 00:07:00.523211 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":61,"threads_started":1}
I20260812 00:07:00.523509 21871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2338697d94c4392ba9e7ba13453462a" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" is_pre_election: true
I20260812 00:07:00.523713 21871 raft_consensus.cc:2454] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 0.
I20260812 00:07:00.523665 21348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2338697d94c4392ba9e7ba13453462a" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330181aaf8134023a96c5e42e73e08b0" is_pre_election: true
I20260812 00:07:00.523809 21348 raft_consensus.cc:2454] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 0.
I20260812 00:07:00.524008 21490 leader_election.cc:302] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a23078d725d4b9e8dc6d6900e3ae6f0, b72df70ea65b44d9bbcef5d76e3983db; no voters:
I20260812 00:07:00.524283 22039 raft_consensus.cc:2781] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:00.524385 22039 raft_consensus.cc:482] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:00.524420 22039 raft_consensus.cc:3037] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:00.525408 22039 raft_consensus.cc:504] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.525584 22039 leader_election.cc:288] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 election: Requested vote from peers 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603), 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:00.525755 21348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2338697d94c4392ba9e7ba13453462a" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330181aaf8134023a96c5e42e73e08b0"
I20260812 00:07:00.525872 21348 raft_consensus.cc:3037] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:00.525926 21871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2338697d94c4392ba9e7ba13453462a" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0"
I20260812 00:07:00.526002 21871 raft_consensus.cc:3037] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:00.526669 21348 raft_consensus.cc:2454] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 1.
I20260812 00:07:00.526896 21871 raft_consensus.cc:2454] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 1.
I20260812 00:07:00.526909 21489 leader_election.cc:302] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330181aaf8134023a96c5e42e73e08b0, b72df70ea65b44d9bbcef5d76e3983db; no voters:
I20260812 00:07:00.527026 22039 raft_consensus.cc:2781] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:00.527195 22039 raft_consensus.cc:686] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 1 LEADER]: Becoming Leader. State: Replica: b72df70ea65b44d9bbcef5d76e3983db, State: Running, Role: LEADER
I20260812 00:07:00.527312 22039 consensus_queue.cc:237] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:00.528472 19487 catalog_manager.cc:5495] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db reported cstate change: term changed from 0 to 1, leader changed from <none> to b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194). New cstate: current_term: 1 leader_uuid: "b72df70ea65b44d9bbcef5d76e3983db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } health_report { overall_health: UNKNOWN } } }
I20260812 00:07:00.551097 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.548027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.7.85.194:46105)
0812 00:07:00.548169 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.548171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.548183 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:07:00.548306 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.548307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.548332 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.548405 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:00.548617 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.549101 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.550101 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.550298 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.550364 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.550678 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:00.550815 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:00.550891 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.550950 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.550970 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 00:07:00.553803 21348 raft_consensus.cc:1261] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Refusing update from remote peer b72df70ea65b44d9bbcef5d76e3983db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:00.553803 21871 raft_consensus.cc:1261] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Refusing update from remote peer b72df70ea65b44d9bbcef5d76e3983db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:00.554273 22039 consensus_queue.cc:1035] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:00.554503 22046 consensus_queue.cc:1035] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [LEADER]: Connected to new peer: Peer: permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:00.558327 22059 mvcc.cc:205] Tried to move back new op lower bound from 7317476231383896064 to 7317476231280816128. Current Snapshot: MvccSnapshot[applied={T|T < 7317476231383896064}]
I20260812 00:07:00.558456 22055 mvcc.cc:205] Tried to move back new op lower bound from 7317476231383896064 to 7317476231280816128. Current Snapshot: MvccSnapshot[applied={T|T < 7317476231383896064}]
I20260812 00:07:00.563393 19489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:00.563633 19489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:00.566011 19489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c79758a37f514ac292fd19a499799159: Sending DeleteTablet for 3 replicas of tablet b2338697d94c4392ba9e7ba13453462a
I20260812 00:07:00.566387 21310 tablet_service.cc:1514] Processing DeleteTablet for tablet b2338697d94c4392ba9e7ba13453462a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:00 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:54954
I20260812 00:07:00.567253 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet b2338697d94c4392ba9e7ba13453462a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:00 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:52442
I20260812 00:07:00.567507 22073 tablet_replica.cc:330] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: stopping tablet replica
I20260812 00:07:00.567709 22073 raft_consensus.cc:2229] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:00.567739 21573 tablet_service.cc:1514] Processing DeleteTablet for tablet b2338697d94c4392ba9e7ba13453462a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:00 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:47036
I20260812 00:07:00.568095 22076 tablet_replica.cc:330] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: stopping tablet replica
I20260812 00:07:00.568195 22076 raft_consensus.cc:2229] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:00.568306 22076 raft_consensus.cc:2258] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:00.568856 22076 ts_tablet_manager.cc:1905] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:00.570066 22073 raft_consensus.cc:2258] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:00.570611 22073 ts_tablet_manager.cc:1905] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:00.571197 22076 ts_tablet_manager.cc:1918] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:00.571267 22076 log.cc:1196] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/b2338697d94c4392ba9e7ba13453462a
I20260812 00:07:00.571558 22076 ts_tablet_manager.cc:1939] T b2338697d94c4392ba9e7ba13453462a P b72df70ea65b44d9bbcef5d76e3983db: Deleting consensus metadata
I20260812 00:07:00.572110 19470 catalog_manager.cc:4841] TS b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194:46105): tablet b2338697d94c4392ba9e7ba13453462a (table test-table [id=c7c2c3a208744dfb9874d3168cfa03a3]) successfully deleted
I20260812 00:07:00.572403 22073 ts_tablet_manager.cc:1918] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:00.572479 22073 log.cc:1196] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/b2338697d94c4392ba9e7ba13453462a
I20260812 00:07:00.572738 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 19429
I20260812 00:07:00.572793 22073 ts_tablet_manager.cc:1939] T b2338697d94c4392ba9e7ba13453462a P 6a23078d725d4b9e8dc6d6900e3ae6f0: Deleting consensus metadata
I20260812 00:07:00.593838 22072 tablet_replica.cc:330] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: stopping tablet replica
I20260812 00:07:00.594179 22072 raft_consensus.cc:2229] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:00.594432 22072 raft_consensus.cc:2258] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:00.595070 22072 ts_tablet_manager.cc:1905] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:00.597875 22072 ts_tablet_manager.cc:1918] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:00.598138 22072 log.cc:1196] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/b2338697d94c4392ba9e7ba13453462a
I20260812 00:07:00.599445 22072 ts_tablet_manager.cc:1939] T b2338697d94c4392ba9e7ba13453462a P 330181aaf8134023a96c5e42e73e08b0: Deleting consensus metadata
W20260812 00:07:00.600683 21219 connection.cc:535] server connection from 127.0.0.1:54954 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 00:07:00.601253 21728 connection.cc:535] server connection from 127.0.0.1:52442 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 00:07:00.603834 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.571654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.7.85.252:35855)
0812 00:07:00.571704 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.571706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.572779 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 00:07:00.593681 (+ 20902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.593684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.593788 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.593889 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.594905 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.595763 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.596956 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.597393 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.597445 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.599423 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.599425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.599426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.599436 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.602015 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.602449 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.602450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.602451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.602521 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.602752 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.602753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.602753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.602900 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.603010 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.603529 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.603718 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:07:00.612528 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.603945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.7.85.252:35855)
0812 00:07:00.604383 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.604385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.604493 (+ 108us) server_negotiation.cc:407] Connection header received
0812 00:07:00.604553 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.604554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.604565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.604624 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.605366 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.606095 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.607206 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.607445 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.607512 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.609217 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.609220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.609220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.609230 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.610865 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.611259 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.611261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.611262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.611329 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.611549 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.611550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.611551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.611726 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.612057 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.612157 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.612325 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
I20260812 00:07:00.614782 21979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:00.603601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.7.85.253:43829)
0812 00:07:00.603676 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:00.603678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:00.604080 (+ 402us) server_negotiation.cc:407] Connection header received
0812 00:07:00.604179 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:00.604180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:00.604198 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:00.604229 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:00.605275 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.606095 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.607422 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:00.607688 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:00.608138 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:00.610298 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:00.610300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:00.610300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:00.610326 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:00.612535 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.613348 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.613350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.613351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.613418 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:00.613762 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:00.613764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:00.613764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:00.614083 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:00.614195 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:00.614481 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 00:07:00.614626 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:07:01.560052 22097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:01.559707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:40831)
0812 00:07:01.559922 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:01.560008 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260812 00:07:01.560051 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:01.559706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:34495)
0812 00:07:01.559877 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:01.559985 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
W20260812 00:07:01.560356 21676 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:43135 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
W20260812 00:07:01.560496 21952 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:43135 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:01.561265 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:01.560706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:48395)
0812 00:07:01.561126 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:01.561222 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":69,"threads_started":1}
W20260812 00:07:01.561517 21439 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:43135 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:02.066090 22101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c79758a37f514ac292fd19a499799159)
I20260812 00:07:02.066247 22101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:02.066326 22102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c79758a37f514ac292fd19a499799159)
I20260812 00:07:02.066424 22102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:02.066462 22101 leader_election.cc:288] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252:35855)
I20260812 00:07:02.066630 22102 leader_election.cc:288] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253:43829)
I20260812 00:07:02.067098 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.066812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57466)
0812 00:07:02.066965 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.067042 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260812 00:07:02.067127 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.066571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57460)
0812 00:07:02.067010 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.067075 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":144,"threads_started":1}
W20260812 00:07:02.067256 21047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
W20260812 00:07:02.067472 20622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:02.067754 22106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.067666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57472)
0812 00:07:02.067704 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.067728 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 00:07:02.067830 21047 leader_election.cc:334] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:02.068204 22109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.068089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57484)
0812 00:07:02.068147 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.068179 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 00:07:02.068370 20622 leader_election.cc:334] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:02.070197 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.067046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.7.85.252:35855)
0812 00:07:02.067129 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:02.067132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:02.067252 (+ 120us) server_negotiation.cc:407] Connection header received
0812 00:07:02.067359 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:02.067360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:02.067370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:02.067438 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:02.067653 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.068367 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.069435 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.069820 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.069913 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:02.069940 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:02.070087 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 00:07:02.070128 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":45,"threads_started":1}
I20260812 00:07:02.070199 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.066852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:35855 (local address 127.0.0.1:39076)
0812 00:07:02.067188 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.067199 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:02.067281 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:02.067515 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:02.067516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:02.067518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:02.067570 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:02.067572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.068475 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.068476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:02.069338 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:02.069340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.069917 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.069918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:02.069985 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:02.069996 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 00:07:02.070080 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:07:02.070126 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":49,"threads_started":1}
I20260812 00:07:02.070572 21104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13529a78b39e49808ed2fc0c855410f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9da968a3e1db4b16b7270f7fa114f67f" is_pre_election: true
I20260812 00:07:02.070660 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.066789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:43829 (local address 127.0.0.1:55270)
0812 00:07:02.066960 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.066974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:02.067063 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:02.067380 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:02.067382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:02.067385 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:02.067452 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:02.067454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.068551 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.068552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:02.069469 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:02.069472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.070132 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.070134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:02.070207 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:02.070210 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:07:02.070463 (+ 253us) client_negotiation.cc:240] Negotiation successful
0812 00:07:02.070514 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260812 00:07:02.070720 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.066845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.7.85.253:43829)
0812 00:07:02.067175 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:02.067177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:02.067186 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:02.067215 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:02.067216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:02.067228 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:02.067319 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:02.067540 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.068458 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.069568 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:02.070020 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:02.070127 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:02.070141 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:02.070592 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 00:07:02.070643 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":60,"threads_started":1}
I20260812 00:07:02.070746 21104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13529a78b39e49808ed2fc0c855410f7 in term 1.
I20260812 00:07:02.070926 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9da968a3e1db4b16b7270f7fa114f67f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "13529a78b39e49808ed2fc0c855410f7" is_pre_election: true
I20260812 00:07:02.070947 20622 leader_election.cc:302] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13529a78b39e49808ed2fc0c855410f7, 9da968a3e1db4b16b7270f7fa114f67f; no voters: c79758a37f514ac292fd19a499799159
I20260812 00:07:02.071069 20669 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9da968a3e1db4b16b7270f7fa114f67f in term 1.
I20260812 00:07:02.071125 22101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 00:07:02.071184 22101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c79758a37f514ac292fd19a499799159)
I20260812 00:07:02.071223 22101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:07:02.071249 21043 leader_election.cc:302] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13529a78b39e49808ed2fc0c855410f7, 9da968a3e1db4b16b7270f7fa114f67f; no voters: c79758a37f514ac292fd19a499799159
I20260812 00:07:02.071381 22102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 00:07:02.071429 22102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Starting leader election (detected failure of leader c79758a37f514ac292fd19a499799159)
I20260812 00:07:02.071460 22102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:07:02.072099 22101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:02.072257 22101 leader_election.cc:288] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 election: Requested vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252:35855)
I20260812 00:07:02.072264 22102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:02.072431 22102 leader_election.cc:288] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 2 election: Requested vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 13529a78b39e49808ed2fc0c855410f7 (127.7.85.253:43829)
I20260812 00:07:02.072472 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.072355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57490)
0812 00:07:02.072400 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.072442 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 00:07:02.072506 21104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13529a78b39e49808ed2fc0c855410f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9da968a3e1db4b16b7270f7fa114f67f"
I20260812 00:07:02.072610 21104 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 13529a78b39e49808ed2fc0c855410f7 in current term 2: Already voted for candidate 9da968a3e1db4b16b7270f7fa114f67f in this term.
I20260812 00:07:02.072685 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.072583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57494)
0812 00:07:02.072621 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.072651 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 00:07:02.072759 20669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9da968a3e1db4b16b7270f7fa114f67f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "13529a78b39e49808ed2fc0c855410f7"
I20260812 00:07:02.072839 20669 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9da968a3e1db4b16b7270f7fa114f67f in current term 2: Already voted for candidate 13529a78b39e49808ed2fc0c855410f7 in this term.
I20260812 00:07:02.073024 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.072909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57506)
0812 00:07:02.072965 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.072991 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 00:07:02.073082 21047 leader_election.cc:334] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:02.073113 21047 leader_election.cc:302] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9da968a3e1db4b16b7270f7fa114f67f; no voters: 13529a78b39e49808ed2fc0c855410f7, c79758a37f514ac292fd19a499799159
I20260812 00:07:02.073132 22103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.073016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:57512)
0812 00:07:02.073057 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.073103 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 00:07:02.073232 22102 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 00:07:02.073232 20622 leader_election.cc:334] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:02.073278 20622 leader_election.cc:302] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13529a78b39e49808ed2fc0c855410f7; no voters: 9da968a3e1db4b16b7270f7fa114f67f, c79758a37f514ac292fd19a499799159
I20260812 00:07:02.073350 22101 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 00:07:02.561429 22122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.561122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:60597)
0812 00:07:02.561308 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.561360 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260812 00:07:02.561432 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.561021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:55849)
0812 00:07:02.561308 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.561362 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":88,"threads_started":1}
I20260812 00:07:02.562060 22124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:02.561828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:52589)
0812 00:07:02.561966 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:02.562010 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":54,"threads_started":1}
I20260812 00:07:03.562667 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.562210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:33725)
0812 00:07:03.562547 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.562595 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":94,"threads_started":1}
I20260812 00:07:03.562884 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.562616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:49973)
0812 00:07:03.562778 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.562822 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
I20260812 00:07:03.562888 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.562226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:54081)
0812 00:07:03.562415 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.562825 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":82,"threads_started":1}
I20260812 00:07:03.936333 22129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:03.936470 22129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:03.936748 22129 leader_election.cc:288] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252:35855)
I20260812 00:07:03.937388 21104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13529a78b39e49808ed2fc0c855410f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9da968a3e1db4b16b7270f7fa114f67f" is_pre_election: true
I20260812 00:07:03.937440 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.937000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:49436)
0812 00:07:03.937292 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.937367 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":86,"threads_started":1}
I20260812 00:07:03.937502 21104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13529a78b39e49808ed2fc0c855410f7 in term 2.
I20260812 00:07:03.937736 20622 leader_election.cc:302] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13529a78b39e49808ed2fc0c855410f7, 9da968a3e1db4b16b7270f7fa114f67f; no voters:
I20260812 00:07:03.937860 22129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 00:07:03.937922 22129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:03.937951 22129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:07:03.938155 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.937935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:49442)
0812 00:07:03.938056 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.938110 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260812 00:07:03.938290 20622 leader_election.cc:334] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:03.938941 22129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:03.939096 22129 leader_election.cc:288] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 3 election: Requested vote from peers c79758a37f514ac292fd19a499799159 (127.7.85.254:43135), 9da968a3e1db4b16b7270f7fa114f67f (127.7.85.252:35855)
I20260812 00:07:03.939409 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.939198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:49446)
0812 00:07:03.939269 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.939345 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:07:03.939462 21104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "13529a78b39e49808ed2fc0c855410f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9da968a3e1db4b16b7270f7fa114f67f"
I20260812 00:07:03.939688 21104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:07:03.939958 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.939837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:49454)
0812 00:07:03.939883 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.939919 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 00:07:03.940079 20622 leader_election.cc:334] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
I20260812 00:07:03.940672 21104 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13529a78b39e49808ed2fc0c855410f7 in term 3.
I20260812 00:07:03.941067 20622 leader_election.cc:302] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 13529a78b39e49808ed2fc0c855410f7, 9da968a3e1db4b16b7270f7fa114f67f; no voters: c79758a37f514ac292fd19a499799159
I20260812 00:07:03.941342 22129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 00:07:03.941499 22129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [term 3 LEADER]: Becoming Leader. State: Replica: 13529a78b39e49808ed2fc0c855410f7, State: Running, Role: LEADER
I20260812 00:07:03.941596 22129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } }
I20260812 00:07:03.942013 22135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13529a78b39e49808ed2fc0c855410f7. Latest consensus state: current_term: 3 leader_uuid: "13529a78b39e49808ed2fc0c855410f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:03.942127 22135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:03.942323 22141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:07:03.942929 22141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:03.943041 22141 catalog_manager.cc:1259] Loaded cluster ID: bca890f5cf0f40b0a221b08e88d5662f
I20260812 00:07:03.943078 22141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:07:03.943331 22141 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:03.943444 22141 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Loaded TSK: 0
I20260812 00:07:03.943600 22141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:07:03.966527 20649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:07:03.968266 20649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:03.969415 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.969280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:49474)
0812 00:07:03.969340 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.969377 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:07:03.969548 21104 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [term 3 FOLLOWER]: Refusing update from remote peer 13529a78b39e49808ed2fc0c855410f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 00:07:03.969836 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.969720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.0.0.1:49490)
0812 00:07:03.969783 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.969816 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:07:03.969830 22129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 00:07:03.969986 20622 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 -> Peer c79758a37f514ac292fd19a499799159 (127.7.85.254:43135): Couldn't send request to peer c79758a37f514ac292fd19a499799159. Status: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 00:07:03.970497 22146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13529a78b39e49808ed2fc0c855410f7. Latest consensus state: current_term: 3 leader_uuid: "13529a78b39e49808ed2fc0c855410f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:03.970619 22146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:07:03.971457 22129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "13529a78b39e49808ed2fc0c855410f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:03.971762 22129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:03.972102 22146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "13529a78b39e49808ed2fc0c855410f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c79758a37f514ac292fd19a499799159" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43135 } } peers { permanent_uuid: "13529a78b39e49808ed2fc0c855410f7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 43829 } attrs { promote: false } } peers { permanent_uuid: "9da968a3e1db4b16b7270f7fa114f67f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 35855 } attrs { promote: false } } }
I20260812 00:07:03.972277 22146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da968a3e1db4b16b7270f7fa114f67f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:07:03.976897 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.973345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:46105 (local address 127.0.0.1:53636)
0812 00:07:03.973549 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.973562 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:03.973623 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:03.973968 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:03.973969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:03.973972 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:03.974046 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:03.974049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.975013 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.975014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:03.976012 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:03.976015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.976628 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.976629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:03.976690 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:03.976696 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:07:03.976756 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 00:07:03.976800 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":77}
I20260812 00:07:03.976902 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.973624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.7.85.194:46105)
0812 00:07:03.973764 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:03.973766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:03.973776 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:03.973824 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:03.973825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:03.973839 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:03.973908 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:03.974115 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.974905 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.976080 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.976538 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.976614 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:03.976628 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:03.976777 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 00:07:03.976821 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260812 00:07:03.977473 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.973365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:36111 (local address 127.0.0.1:56828)
0812 00:07:03.973707 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.973716 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:03.973753 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:03.974095 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:03.974096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:03.974099 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:03.974147 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:03.974148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.975043 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.975044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:03.976378 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:03.976380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.977233 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.977234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:03.977296 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:03.977301 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:03.977340 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 00:07:03.977391 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":133,"thread_start_us":121,"threads_started":1}
I20260812 00:07:03.977562 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.973365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40603 (local address 127.0.0.1:47142)
0812 00:07:03.973936 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.973947 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:03.974002 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:03.974220 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:03.974221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:03.974222 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:03.974257 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:03.974258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.975068 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.975068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:03.976378 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:03.976380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.977247 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.977248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:03.977321 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:03.977324 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:07:03.977401 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 00:07:03.977452 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":82,"thread_start_us":135,"threads_started":1}
I20260812 00:07:03.977571 21573 tablet_service.cc:1467] Processing CreateTablet for tablet 92ccf2cf444549b18c625a6330d3c089 (DEFAULT_TABLE table=test-table [id=72a96cc659bf44b3bd4b974176854917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:03.977735 21573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ccf2cf444549b18c625a6330d3c089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:03.977834 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.973804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.7.85.195:36111)
0812 00:07:03.973921 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:03.973923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:03.973930 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:07:03.973958 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:03.973959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:03.973971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:03.974041 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:03.974239 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.974955 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.976485 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.977102 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.977669 (+ 567us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:03.977698 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:03.977717 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 00:07:03.977756 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260812 00:07:03.978044 21832 tablet_service.cc:1467] Processing CreateTablet for tablet 92ccf2cf444549b18c625a6330d3c089 (DEFAULT_TABLE table=test-table [id=72a96cc659bf44b3bd4b974176854917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:03.978180 21832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ccf2cf444549b18c625a6330d3c089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:03.979153 22166 tablet_bootstrap.cc:492] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Bootstrap starting.
I20260812 00:07:03.979243 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.973646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.7.85.193:40603)
0812 00:07:03.974095 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:03.974097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:03.974105 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:03.974131 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:03.974132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:03.974143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:03.974205 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:03.974316 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.974940 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.976477 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:03.977081 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:03.977244 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:03.977258 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:03.977411 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:07:03.977452 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":95,"threads_started":1}
I20260812 00:07:03.979525 22167 tablet_bootstrap.cc:492] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Bootstrap starting.
I20260812 00:07:03.979643 21310 tablet_service.cc:1467] Processing CreateTablet for tablet 92ccf2cf444549b18c625a6330d3c089 (DEFAULT_TABLE table=test-table [id=72a96cc659bf44b3bd4b974176854917]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:03.979775 21310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ccf2cf444549b18c625a6330d3c089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:03.979809 22166 tablet_bootstrap.cc:654] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:03.979964 22167 tablet_bootstrap.cc:654] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:03.980484 22166 tablet_bootstrap.cc:492] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: No bootstrap required, opened a new log
I20260812 00:07:03.980525 22166 ts_tablet_manager.cc:1397] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:03.980590 22167 tablet_bootstrap.cc:492] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: No bootstrap required, opened a new log
I20260812 00:07:03.980641 22167 ts_tablet_manager.cc:1397] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:03.980618 22166 raft_consensus.cc:348] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:03.980670 22166 raft_consensus.cc:374] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:03.980684 22166 raft_consensus.cc:729] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b72df70ea65b44d9bbcef5d76e3983db, State: Initialized, Role: FOLLOWER
I20260812 00:07:03.980723 22166 consensus_queue.cc:260] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:03.980903 22166 ts_tablet_manager.cc:1428] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:07:03.981364 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.981127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:44375)
0812 00:07:03.981283 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.981324 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260812 00:07:03.981690 22167 raft_consensus.cc:348] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:03.981782 22167 raft_consensus.cc:374] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:03.981801 22167 raft_consensus.cc:729] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a23078d725d4b9e8dc6d6900e3ae6f0, State: Initialized, Role: FOLLOWER
I20260812 00:07:03.982079 22167 consensus_queue.cc:260] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:03.982154 22174 tablet_bootstrap.cc:492] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0: Bootstrap starting.
I20260812 00:07:03.982316 22167 ts_tablet_manager.cc:1428] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:03.982550 22174 tablet_bootstrap.cc:654] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:03.982610 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.982498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:52601)
0812 00:07:03.982542 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.982579 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 00:07:03.982991 21956 heartbeater.cc:503] Master 127.7.85.253:43829 was elected leader, sending a full tablet report...
I20260812 00:07:03.983165 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.983009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:47949)
0812 00:07:03.983107 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.983134 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 00:07:03.983498 22174 tablet_bootstrap.cc:492] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0: No bootstrap required, opened a new log
I20260812 00:07:03.983572 22174 ts_tablet_manager.cc:1397] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:03.983773 22174 raft_consensus.cc:348] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:03.983856 22174 raft_consensus.cc:374] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:03.983882 22174 raft_consensus.cc:729] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330181aaf8134023a96c5e42e73e08b0, State: Initialized, Role: FOLLOWER
I20260812 00:07:03.983939 22174 consensus_queue.cc:260] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:03.984122 22174 ts_tablet_manager.cc:1428] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:03.984517 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:03.984411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:37825)
0812 00:07:03.984448 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:03.984485 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 00:07:03.984870 21444 heartbeater.cc:503] Master 127.7.85.253:43829 was elected leader, sending a full tablet report...
I20260812 00:07:04.108428 22173 raft_consensus.cc:482] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:04.108551 22173 raft_consensus.cc:504] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:04.108743 22173 leader_election.cc:288] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603), 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:04.108970 21348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ccf2cf444549b18c625a6330d3c089" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330181aaf8134023a96c5e42e73e08b0" is_pre_election: true
I20260812 00:07:04.109071 21348 raft_consensus.cc:2454] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 0.
I20260812 00:07:04.109035 21871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ccf2cf444549b18c625a6330d3c089" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" is_pre_election: true
I20260812 00:07:04.109226 21871 raft_consensus.cc:2454] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 0.
I20260812 00:07:04.109354 21489 leader_election.cc:302] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330181aaf8134023a96c5e42e73e08b0, b72df70ea65b44d9bbcef5d76e3983db; no voters:
I20260812 00:07:04.109468 22173 raft_consensus.cc:2781] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:04.109525 22173 raft_consensus.cc:482] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:04.109540 22173 raft_consensus.cc:3037] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:04.110312 22173 raft_consensus.cc:504] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:04.110461 22173 leader_election.cc:288] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 election: Requested vote from peers 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603), 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111)
I20260812 00:07:04.110637 21348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ccf2cf444549b18c625a6330d3c089" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "330181aaf8134023a96c5e42e73e08b0"
I20260812 00:07:04.110662 21871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ccf2cf444549b18c625a6330d3c089" candidate_uuid: "b72df70ea65b44d9bbcef5d76e3983db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0"
I20260812 00:07:04.110728 21348 raft_consensus.cc:3037] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:04.110735 21871 raft_consensus.cc:3037] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:04.111534 21871 raft_consensus.cc:2454] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 1.
I20260812 00:07:04.111534 21348 raft_consensus.cc:2454] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b72df70ea65b44d9bbcef5d76e3983db in term 1.
I20260812 00:07:04.111759 21489 leader_election.cc:302] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 330181aaf8134023a96c5e42e73e08b0, b72df70ea65b44d9bbcef5d76e3983db; no voters:
I20260812 00:07:04.111938 22173 raft_consensus.cc:2781] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:04.112195 22173 raft_consensus.cc:686] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 1 LEADER]: Becoming Leader. State: Replica: b72df70ea65b44d9bbcef5d76e3983db, State: Running, Role: LEADER
I20260812 00:07:04.112285 22173 consensus_queue.cc:237] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } }
I20260812 00:07:04.112483 21682 heartbeater.cc:503] Master 127.7.85.253:43829 was elected leader, sending a full tablet report...
I20260812 00:07:04.112674 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.112519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:46065)
0812 00:07:04.112584 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.112625 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 00:07:04.113030 20652 catalog_manager.cc:5495] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db reported cstate change: term changed from 0 to 1, leader changed from <none> to b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194). New cstate: current_term: 1 leader_uuid: "b72df70ea65b44d9bbcef5d76e3983db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b72df70ea65b44d9bbcef5d76e3983db" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 46105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 } health_report { overall_health: UNKNOWN } } }
I20260812 00:07:04.127305 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.124172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.7.85.194:46105)
0812 00:07:04.124329 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:04.124332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:04.124355 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:07:04.124492 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:04.124495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:04.124520 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:04.124584 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:04.124791 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:04.125400 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:04.126186 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:04.126458 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:04.126524 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:04.126761 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:04.126817 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:04.126915 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:04.127080 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 00:07:04.127149 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 00:07:04.128552 21871 raft_consensus.cc:1261] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Refusing update from remote peer b72df70ea65b44d9bbcef5d76e3983db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:04.128558 21348 raft_consensus.cc:1261] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Refusing update from remote peer b72df70ea65b44d9bbcef5d76e3983db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:04.128801 22181 consensus_queue.cc:1035] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a23078d725d4b9e8dc6d6900e3ae6f0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:04.129014 22173 consensus_queue.cc:1035] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [LEADER]: Connected to new peer: Peer: permanent_uuid: "330181aaf8134023a96c5e42e73e08b0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 40603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:04.129597 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.129467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:46723)
0812 00:07:04.129528 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.129562 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:07:04.130128 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.129991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:53887)
0812 00:07:04.130050 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.130082 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 00:07:04.130738 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.130592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:43203)
0812 00:07:04.130666 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.130691 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 00:07:04.131532 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.131244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.195:41545)
0812 00:07:04.131463 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.131495 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260812 00:07:04.131661 22125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.131555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.194:53833)
0812 00:07:04.131590 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.131611 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 00:07:04.132428 22190 mvcc.cc:205] Tried to move back new op lower bound from 7317476246027915264 to 7317476245964566528. Current Snapshot: MvccSnapshot[applied={T|T < 7317476246027915264}]
I20260812 00:07:04.132700 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.132020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43135 (local address 127.7.85.193:35145)
0812 00:07:04.132640 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.132672 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:43135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603}
I20260812 00:07:04.133093 20649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:04.133231 20649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:04.134802 20649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13529a78b39e49808ed2fc0c855410f7: Sending DeleteTablet for 3 replicas of tablet 92ccf2cf444549b18c625a6330d3c089
I20260812 00:07:04.135135 21832 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ccf2cf444549b18c625a6330d3c089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:04 UTC) from {username='oryx', principal='oryx/127.7.85.253@KRBTEST.COM'} at 127.0.0.1:56828
I20260812 00:07:04.135138 21573 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ccf2cf444549b18c625a6330d3c089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:04 UTC) from {username='oryx', principal='oryx/127.7.85.253@KRBTEST.COM'} at 127.0.0.1:53636
I20260812 00:07:04.135218 21310 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ccf2cf444549b18c625a6330d3c089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:04 UTC) from {username='oryx', principal='oryx/127.7.85.253@KRBTEST.COM'} at 127.0.0.1:47142
I20260812 00:07:04.135404 22204 tablet_replica.cc:330] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: stopping tablet replica
I20260812 00:07:04.135474 22204 raft_consensus.cc:2229] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:04.135468 22205 tablet_replica.cc:330] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: stopping tablet replica
I20260812 00:07:04.135527 22205 raft_consensus.cc:2229] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:04.135555 22204 raft_consensus.cc:2258] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:04.135624 22206 tablet_replica.cc:330] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0: stopping tablet replica
I20260812 00:07:04.135703 22206 raft_consensus.cc:2229] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:04.135730 22205 raft_consensus.cc:2258] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:04.135771 22206 raft_consensus.cc:2258] T 92ccf2cf444549b18c625a6330d3c089 P 330181aaf8134023a96c5e42e73e08b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:04.135869 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 20998
I20260812 00:07:04.135990 22204 ts_tablet_manager.cc:1905] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:04.138450 22204 ts_tablet_manager.cc:1918] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:04.138541 22204 log.cc:1196] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/92ccf2cf444549b18c625a6330d3c089
I20260812 00:07:04.138852 22204 ts_tablet_manager.cc:1939] T 92ccf2cf444549b18c625a6330d3c089 P 6a23078d725d4b9e8dc6d6900e3ae6f0: Deleting consensus metadata
I20260812 00:07:04.139407 20616 catalog_manager.cc:4841] TS 6a23078d725d4b9e8dc6d6900e3ae6f0 (127.7.85.195:36111): tablet 92ccf2cf444549b18c625a6330d3c089 (table test-table [id=72a96cc659bf44b3bd4b974176854917]) successfully deleted
I20260812 00:07:04.149709 22205 ts_tablet_manager.cc:1905] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:04.152127 22205 ts_tablet_manager.cc:1918] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:04.152235 22205 log.cc:1196] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/92ccf2cf444549b18c625a6330d3c089
I20260812 00:07:04.152554 22205 ts_tablet_manager.cc:1939] T 92ccf2cf444549b18c625a6330d3c089 P b72df70ea65b44d9bbcef5d76e3983db: Deleting consensus metadata
I20260812 00:07:04.153569 20624 catalog_manager.cc:4841] TS b72df70ea65b44d9bbcef5d76e3983db (127.7.85.194:46105): tablet 92ccf2cf444549b18c625a6330d3c089 (table test-table [id=72a96cc659bf44b3bd4b974176854917]) successfully deleted
I20260812 00:07:04.157846 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 21459
I20260812 00:07:04.157980 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:04.157829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:40603 (local address 127.0.0.1:47144)
0812 00:07:04.157892 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:04.157929 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:40603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 00:07:04.172685 20612 catalog_manager.cc:4584] TS 330181aaf8134023a96c5e42e73e08b0 (127.7.85.193:40603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92ccf2cf444549b18c625a6330d3c089: Network error: Client connection negotiation failed: client connection to 127.7.85.193:40603: connect: Connection refused (error 111)
I20260812 00:07:04.179028 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 21695
I20260812 00:07:04.204169 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 20397
I20260812 00:07:04.220377 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 20767
2026-08-12T00:07:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7525 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:07:04 dist-test-slave-z8x9 krb5kdc[22211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:07:04 dist-test-slave-z8x9 krb5kdc[22211](info): set up 2 sockets
Aug 12 00:07:04 dist-test-slave-z8x9 krb5kdc[22211](info): commencing operation
krb5kdc: starting...
W20260812 00:07:05.596067 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:07:05 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab.
Entry for principal oryx/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab.
I20260812 00:07:05.744674 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41095
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab
--principal=oryx/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:05.838547 22236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:05.838807 22236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:05.838833 22236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:05.840123 22236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:05.840202 22236 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 00:07:05.840219 22236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:05.840237 22236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:05.840250 22236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:05.840291 22236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:05.842101 22236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41095
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.254.keytab
--principal=oryx/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:05.842442 22236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:05.842720 22236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:05.849135 22248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:05.849120 22250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:05.849100 22252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:05.849231 22236 server_base.cc:1034] running on GCE node
I20260812 00:07:05.849682 22236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:05.849980 22236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:05.851151 22236 hybrid_clock.cc:648] HybridClock initialized: now 1786493225851138 us; error 31 us; skew 500 ppm
Aug 12 00:07:05 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:05.854009 22236 init.cc:377] Logged in from keytab as oryx/127.7.85.254@KRBTEST.COM (short username oryx)
I20260812 00:07:05.854466 22236 webserver.cc:460] Webserver started at http://127.7.85.254:43813/ using document root <none> and password file <none>
I20260812 00:07:05.854727 22236 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:05.854786 22236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:05.854921 22236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:05.856215 22236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "c4e410903f9e4efab67c0af1bf3c1ac9"
format_stamp: "Formatted at 2026-08-12 00:07:05 on dist-test-slave-z8x9"
server_key: "ca04c5c6e4a080859c153903fc2051af"
server_key_iv: "1c64133a63e517b8277f6c4a963a2880"
server_key_version: "encryptionkey@0"
I20260812 00:07:05.856633 22236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "c4e410903f9e4efab67c0af1bf3c1ac9"
format_stamp: "Formatted at 2026-08-12 00:07:05 on dist-test-slave-z8x9"
server_key: "ca04c5c6e4a080859c153903fc2051af"
server_key_iv: "1c64133a63e517b8277f6c4a963a2880"
server_key_version: "encryptionkey@0"
I20260812 00:07:05.859018 22236 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 00:07:05.860381 22264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:05.860706 22236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:05.860955 22236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "c4e410903f9e4efab67c0af1bf3c1ac9"
format_stamp: "Formatted at 2026-08-12 00:07:05 on dist-test-slave-z8x9"
server_key: "ca04c5c6e4a080859c153903fc2051af"
server_key_iv: "1c64133a63e517b8277f6c4a963a2880"
server_key_version: "encryptionkey@0"
I20260812 00:07:05.861138 22236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:05.879057 22236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:05.881690 22236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:05.881842 22236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:05.886948 22236 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:41095
I20260812 00:07:05.887066 22379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:41095 every 8 connection(s)
I20260812 00:07:05.887447 22236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:07:05.888206 22380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:05.890648 22380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Bootstrap starting.
I20260812 00:07:05.891256 22380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:05.891587 22380 log.cc:824] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:05.891624 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 22236
I20260812 00:07:05.891719 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:07:05.891933 7511 external_mini_cluster.cc:1428] Setting key e02eefecce8aaaafb63f1329d60a7b85
I20260812 00:07:05.892516 22380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: No bootstrap required, opened a new log
I20260812 00:07:05.894964 22380 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41095 } }
I20260812 00:07:05.895115 22380 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:05.895139 22380 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e410903f9e4efab67c0af1bf3c1ac9, State: Initialized, Role: FOLLOWER
I20260812 00:07:05.895318 22380 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41095 } }
I20260812 00:07:05.895432 22380 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:07:05.895480 22380 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:07:05.895555 22380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:05.896157 22380 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41095 } }
I20260812 00:07:05.896287 22380 leader_election.cc:302] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e410903f9e4efab67c0af1bf3c1ac9; no voters:
I20260812 00:07:05.896476 22380 leader_election.cc:288] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:07:05.896539 22390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:05.896711 22390 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [term 1 LEADER]: Becoming Leader. State: Replica: c4e410903f9e4efab67c0af1bf3c1ac9, State: Running, Role: LEADER
I20260812 00:07:05.896770 22380 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:07:05.896909 22390 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41095 } }
I20260812 00:07:05.897589 22392 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41095 } } }
I20260812 00:07:05.897758 22392 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:05.898029 22399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:07:05.898888 22399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:05.898968 22391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e410903f9e4efab67c0af1bf3c1ac9. Latest consensus state: current_term: 1 leader_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e410903f9e4efab67c0af1bf3c1ac9" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41095 } } }
I20260812 00:07:05.899096 22391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:05.900601 22399 catalog_manager.cc:1347] Generated new cluster ID: 6219c9ea988c4919b339af7fcadfb228
I20260812 00:07:05.900667 22399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:07:05 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:05.907724 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:05.892562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50430 (local address 127.7.85.254:41095)
0812 00:07:05.892768 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:05.892779 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:05.892821 (+ 42us) server_negotiation.cc:407] Connection header received
0812 00:07:05.894984 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:05.894994 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:05.895062 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:05.895267 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:05.896157 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:05.898017 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:05.899256 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:05.899783 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:05.899874 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:05.902805 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:05.902810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:05.902811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:05.902824 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:05.905547 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:05.905956 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:05.905959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:05.905960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:05.906081 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:05.906281 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:05.906282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:05.906283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:05.906595 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:05.906745 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:05.907322 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 00:07:05.907495 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260812 00:07:05.908231 22399 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:07:05.908816 22399 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:05.912642 22399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Generated new TSK 0
I20260812 00:07:05.912837 22399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab.
Entry for principal oryx/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab.
I20260812 00:07:05.949664 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41095
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:06.050491 22423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:06.050805 22423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:06.050863 22423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:06.052944 22423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:06.053056 22423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:06.053102 22423 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:07:06.053164 22423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:07:06.055141 22423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:41095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:06.055469 22423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:06.055745 22423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:06.062619 22430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:06.062619 22434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:06.063431 22431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:06.063752 22423 server_base.cc:1034] running on GCE node
I20260812 00:07:06.064175 22423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:06.064519 22423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:06.065743 22423 hybrid_clock.cc:648] HybridClock initialized: now 1786493226065720 us; error 35 us; skew 500 ppm
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:06.069252 22423 init.cc:377] Logged in from keytab as oryx/127.7.85.193@KRBTEST.COM (short username oryx)
I20260812 00:07:06.069814 22423 webserver.cc:460] Webserver started at http://127.7.85.193:46799/ using document root <none> and password file <none>
I20260812 00:07:06.070087 22423 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:06.070163 22423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:06.070323 22423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:06.071460 22423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "61595c509b40003acce95164795b20d0"
server_key_iv: "888cee07f43c0e57eab454ef4b403448"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.071866 22423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "61595c509b40003acce95164795b20d0"
server_key_iv: "888cee07f43c0e57eab454ef4b403448"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.073813 22423 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 00:07:06.075551 22447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.076743 22423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:06.076954 22423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "61595c509b40003acce95164795b20d0"
server_key_iv: "888cee07f43c0e57eab454ef4b403448"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.077095 22423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:06.088734 22423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:06.091708 22423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:06.091907 22423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:06.092504 22423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:06.092566 22423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.092621 22423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:06.092653 22423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:06.106389 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.093146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:57907 (local address 127.7.85.254:41095)
0812 00:07:06.093245 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.093249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.093450 (+ 201us) server_negotiation.cc:407] Connection header received
0812 00:07:06.093959 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.093960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.093978 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.094063 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.094858 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.095383 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.097197 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.097500 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.097701 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.100821 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.100823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.100824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.100839 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.104174 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.104903 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.104905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.104906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.104989 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.105398 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.105400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.105400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.105591 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.105667 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.106139 (+ 472us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.106246 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:07:06.106552 22423 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:42853
I20260812 00:07:06.106585 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.093096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.7.85.193:57907)
0812 00:07:06.093308 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.093352 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.093849 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.094175 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.094178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.094491 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:06.094737 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.094742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.095485 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.095486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.097027 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.097031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.097731 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.097732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.097968 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.098644 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:06.098655 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:06.100432 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:06.104483 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.104494 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.104497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.104807 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.105088 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.105089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.105090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.105303 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.105700 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:06.105709 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:06.105941 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.106082 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.106299 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":120,"threads_started":1}
I20260812 00:07:06.107216 22423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:07:06.107614 22667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:42853 every 8 connection(s)
I20260812 00:07:06.107937 22668 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41095
I20260812 00:07:06.108533 22668 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:06.108753 22668 heartbeater.cc:511] Master 127.7.85.254:41095 requested a full tablet report, sending...
I20260812 00:07:06.109330 22300 ts_manager.cc:194] Registered new tserver with Master: f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853)
I20260812 00:07:06.109962 22300 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:57907
I20260812 00:07:06.115845 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.108334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.7.85.193:51907)
0812 00:07:06.108448 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.108459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.108545 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.108876 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.108878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.109229 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:06.109322 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.109325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.110211 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.110212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.110817 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.110819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.111547 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.111548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.111605 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.112135 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:06.112141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:06.112516 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:06.114353 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.114354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.114355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.114560 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.114876 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.114877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.114877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.114972 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.115514 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:06.115515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:06.115573 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.115652 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.115754 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260812 00:07:06.116207 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.108396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:51907 (local address 127.7.85.254:41095)
0812 00:07:06.108439 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.108441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.108516 (+ 75us) server_negotiation.cc:407] Connection header received
0812 00:07:06.108619 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.108620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.108633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.108698 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.109454 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.110091 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.110981 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.111451 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.111522 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.112639 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.112641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.112641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.112649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.114224 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.114682 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.114684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.114684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.114752 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.115146 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.115148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.115148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.115374 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.115449 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.115825 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.116106 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:07:06.117617 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 22423
I20260812 00:07:06.117746 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:07:06.118001 7511 external_mini_cluster.cc:1428] Setting key 4b73767ab16a2a10e6c37b4e53710afa
WARNING: no policy specified for oryx/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab.
Entry for principal oryx/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab.
I20260812 00:07:06.154381 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41095
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:06.246675 22684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:06.246886 22684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:06.246909 22684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:06.248039 22684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:06.248083 22684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:06.248103 22684 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:07:06.248121 22684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:07:06.249673 22684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:41095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:06.249920 22684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:06.250167 22684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:06.255667 22697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:06.255805 22684 server_base.cc:1034] running on GCE node
W20260812 00:07:06.255677 22693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:06.255702 22692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:06.256173 22684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:06.256451 22684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:06.257624 22684 hybrid_clock.cc:648] HybridClock initialized: now 1786493226257624 us; error 44 us; skew 500 ppm
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:06.260814 22684 init.cc:377] Logged in from keytab as oryx/127.7.85.194@KRBTEST.COM (short username oryx)
I20260812 00:07:06.261514 22684 webserver.cc:460] Webserver started at http://127.7.85.194:41191/ using document root <none> and password file <none>
I20260812 00:07:06.261848 22684 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:06.261924 22684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:06.262094 22684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:06.263195 22684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "374cc386b90d4ff1841e59f7c63635fc"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "0917bb2aed24e5d3a11a44abdf930e46"
server_key_iv: "6df79e18810d31d30d5fab0b360d0505"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.263581 22684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "374cc386b90d4ff1841e59f7c63635fc"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "0917bb2aed24e5d3a11a44abdf930e46"
server_key_iv: "6df79e18810d31d30d5fab0b360d0505"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.265314 22684 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 00:07:06.266188 22710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.266492 22684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:06.266583 22684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "374cc386b90d4ff1841e59f7c63635fc"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "0917bb2aed24e5d3a11a44abdf930e46"
server_key_iv: "6df79e18810d31d30d5fab0b360d0505"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.266680 22684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:06.282061 22684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:06.284842 22684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:06.284978 22684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:06.285604 22684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:06.285681 22684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.285724 22684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:06.285778 22684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.293910 22684 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:33719
I20260812 00:07:06.293989 22917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:33719 every 8 connection(s)
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:06.294332 22684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:07:06.299017 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.286254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.7.85.194:60771)
0812 00:07:06.286454 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.286496 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.287078 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.287412 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.287417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.287806 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:06.288109 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.288116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.289237 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.289239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.290981 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.290988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.291590 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.291591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.291834 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.292987 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:06.292998 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:06.294971 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:06.296886 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.296894 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.296896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.297409 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.297701 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.297702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.297702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.297852 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.298286 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:06.298292 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:06.298560 (+ 268us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.298671 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.298831 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":128,"threads_started":1}
I20260812 00:07:06.299108 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.286288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:60771 (local address 127.7.85.254:41095)
0812 00:07:06.286366 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.286369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.286574 (+ 205us) server_negotiation.cc:407] Connection header received
0812 00:07:06.287221 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.287223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.287239 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.287324 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.288222 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.289042 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.291087 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.291407 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.291507 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.295073 (+ 3566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.295075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.295075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.295083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.296771 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.297538 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.297542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.297542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.297610 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.298028 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.298029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.298029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.298161 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.298230 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.298920 (+ 690us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.298999 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:07:06.299566 22921 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41095
I20260812 00:07:06.299683 22921 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:06.299878 22921 heartbeater.cc:511] Master 127.7.85.254:41095 requested a full tablet report, sending...
I20260812 00:07:06.300381 22298 ts_manager.cc:194] Registered new tserver with Master: 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:06.300957 22298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:60771
I20260812 00:07:06.302673 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 22684
I20260812 00:07:06.302855 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:07:06.303072 7511 external_mini_cluster.cc:1428] Setting key 233d9100c70ecff98b306e81f5b9246c
I20260812 00:07:06.308389 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.300895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:39787 (local address 127.7.85.254:41095)
0812 00:07:06.300960 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.300962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.300971 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:06.301046 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.301047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.301057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.301112 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.301632 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.302630 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.303713 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.304031 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.304085 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.305349 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.305351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.305351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.305358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.307042 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.307441 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.307442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.307443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.307502 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.307764 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.307765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.307766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.307930 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.308006 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.308175 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.308268 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:07:06.308384 22928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.300655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.7.85.194:39787)
0812 00:07:06.300885 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.300896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.300973 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.301124 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.301125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.301379 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:06.301449 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.301451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.302763 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.302765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.303570 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.303574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.304107 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.304108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.304196 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.304700 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:06.304706 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:06.305239 (+ 533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:06.307131 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.307133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.307133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.307360 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.307578 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.307579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.307579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.307644 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.308010 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:06.308011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:06.308065 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.308148 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.308270 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab.
Entry for principal oryx/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab.
I20260812 00:07:06.333076 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:41095
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:06.429601 22932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:06.429843 22932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:06.429870 22932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:06.431097 22932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:06.431138 22932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:06.431160 22932 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:07:06.431177 22932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:06.432732 22932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.85.254:41095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:06.432964 22932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:06.433220 22932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:06.439565 22938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:06.439755 22940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:06.439867 22947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:06.440053 22932 server_base.cc:1034] running on GCE node
I20260812 00:07:06.440622 22932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:06.441009 22932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:06.442229 22932 hybrid_clock.cc:648] HybridClock initialized: now 1786493226442196 us; error 46 us; skew 500 ppm
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:06.445251 22932 init.cc:377] Logged in from keytab as oryx/127.7.85.195@KRBTEST.COM (short username oryx)
I20260812 00:07:06.445744 22932 webserver.cc:460] Webserver started at http://127.7.85.195:35183/ using document root <none> and password file <none>
I20260812 00:07:06.446030 22932 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:06.446086 22932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:06.446228 22932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:06.447359 22932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "229509fd02a349618fe8eb51ead315b9"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "ccb1ff4ed0e903613b737ce222ddf96d"
server_key_iv: "167bdaeceb998ed0ea1dca380ee23d99"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.447731 22932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "229509fd02a349618fe8eb51ead315b9"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "ccb1ff4ed0e903613b737ce222ddf96d"
server_key_iv: "167bdaeceb998ed0ea1dca380ee23d99"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.449389 22932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:07:06.450629 22958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.450906 22932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:06.451017 22932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "229509fd02a349618fe8eb51ead315b9"
format_stamp: "Formatted at 2026-08-12 00:07:06 on dist-test-slave-z8x9"
server_key: "ccb1ff4ed0e903613b737ce222ddf96d"
server_key_iv: "167bdaeceb998ed0ea1dca380ee23d99"
server_key_version: "encryptionkey@0"
I20260812 00:07:06.451102 22932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:06.459740 22932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:06.467478 22932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:06.467665 22932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:06.468300 22932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:06.468341 22932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.468389 22932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:06.468413 22932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:06.477058 22932 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:44807
I20260812 00:07:06.477227 23176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:44807 every 8 connection(s)
I20260812 00:07:06.477751 22932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:07:06.480690 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 22932
I20260812 00:07:06.480808 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:07:06.481021 7511 external_mini_cluster.cc:1428] Setting key e69bd564fac3294b115956c808f7d347
I20260812 00:07:06.481878 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.468801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:35087 (local address 127.7.85.254:41095)
0812 00:07:06.468880 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.468882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.469159 (+ 277us) server_negotiation.cc:407] Connection header received
0812 00:07:06.469762 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.469764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.469779 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.469869 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.470789 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.471640 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.473458 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.473772 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.473852 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.477351 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.477354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.477354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.477370 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.479407 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.480029 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.480031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.480031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.480096 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.480403 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.480489 (+ 86us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.480490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.480688 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.480817 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.481416 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.481781 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:07:06.482137 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.468774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.7.85.195:35087)
0812 00:07:06.469046 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.469076 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.469641 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.469958 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.469964 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.470402 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:06.470691 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.470698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.471773 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.471775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.473345 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.473349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.473924 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.473925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.474107 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.474758 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:06.474784 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:06.477108 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:06.479574 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.479582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.479585 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.479896 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.480179 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.480180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.480180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.480297 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.480850 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:06.480862 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:06.481122 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.481347 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.481855 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":102,"threads_started":1}
I20260812 00:07:06.482833 23177 heartbeater.cc:347] Connected to a master server at 127.7.85.254:41095
I20260812 00:07:06.482940 23177 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:06.483152 23177 heartbeater.cc:511] Master 127.7.85.254:41095 requested a full tablet report, sending...
I20260812 00:07:06.483577 22298 ts_manager.cc:194] Registered new tserver with Master: 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807)
I20260812 00:07:06.484061 22298 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:35087
I20260812 00:07:06.490655 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.484403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:53891 (local address 127.7.85.254:41095)
0812 00:07:06.484479 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.484482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.484663 (+ 181us) server_negotiation.cc:407] Connection header received
0812 00:07:06.484743 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.484744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.484761 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.484812 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.485196 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.485682 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.486434 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.486642 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.486721 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.487809 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.487810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.487811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.487818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.489428 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.489799 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.489801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.489801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.489869 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.490085 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.490086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.490086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.490213 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.490290 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.490467 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.490549 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:07:06.490657 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.484400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.7.85.195:53891)
0812 00:07:06.484568 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.484580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.484687 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.484815 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.484816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.485028 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:06.485091 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.485093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.485796 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.485797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.486338 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.486340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.486734 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.486735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.486804 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.487360 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:06.487366 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:06.487687 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:06.489516 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.489518 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.489518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.489704 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.489946 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:06.489947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:06.489947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:06.490013 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:06.490294 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:06.490295 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:06.490347 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.490427 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.490552 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260812 00:07:06.492889 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 00:07:06.519488 22389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:06.512476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.7.85.254:41095)
0812 00:07:06.512582 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.512585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.512593 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:06.512629 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.512630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.512641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.512733 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.513439 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.513974 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.514801 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.514994 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.515071 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.519372 (+ 4301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50434: BlockingRecv error: recv got EOF from 127.0.0.1:50434 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 00:07:06 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493226, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:06.529552 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.520732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.7.85.254:41095)
0812 00:07:06.520801 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.520804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.520812 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:06.520841 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.520846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.520856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.520915 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:06.521873 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.522675 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.523507 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.523782 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.523867 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.526414 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:06.526415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:06.526416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:06.526424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:06.528258 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.528668 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.528670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.528670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.528740 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:06.528954 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:06.528955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:06.528955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:06.529082 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:06.529219 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.529351 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.529441 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:07:06.531805 22298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:07:06.533609 22298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:06.540644 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.536867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.7.85.193:42853)
0812 00:07:06.537086 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.537090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.537102 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:07:06.537154 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.537157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.537192 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.537273 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:06.537460 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.538534 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.539680 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.540190 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.540285 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.540377 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.540490 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.540531 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":115,"threads_started":1}
I20260812 00:07:06.540640 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.536583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:42853 (local address 127.0.0.1:33714)
0812 00:07:06.536870 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.536880 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.536929 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.537336 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.537337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.537339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:06.537378 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.537379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.538651 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.538652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.539593 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.539596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.540283 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.540285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.540376 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.540394 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.540473 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.540534 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260812 00:07:06.540843 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.536578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.7.85.194:33719)
0812 00:07:06.536739 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.536745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.536789 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:07:06.536900 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.536903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.536919 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.536997 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:06.537306 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.538075 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.539467 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.540226 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.540327 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.540402 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.540679 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.540726 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260812 00:07:06.541070 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.536569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37116 (local address 127.7.85.195:44807)
0812 00:07:06.536971 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.536975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.536986 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:07:06.537027 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.537029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.537047 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.537369 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:06.537421 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.538529 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.539749 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.540465 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.540529 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.540601 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.540936 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.540981 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":81,"threads_started":1}
I20260812 00:07:06.540866 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.536342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44807 (local address 127.0.0.1:37116)
0812 00:07:06.536473 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.536512 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.536585 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.537224 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.537225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.537228 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:06.537290 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.537292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.538625 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.538626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.539662 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.539665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.540634 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.540634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.540692 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.540696 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.540756 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.540794 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
I20260812 00:07:06.541285 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.536377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33719 (local address 127.0.0.1:39126)
0812 00:07:06.536704 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.536714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.536818 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.537100 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.537104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.537106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:06.537216 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.537219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.538170 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.538171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.539388 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.539390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.540441 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.540442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.540527 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.540532 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.540580 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.540634 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":96,"threads_started":1}
I20260812 00:07:06.542042 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 5780ebd3860c47398319ceb79dd568de (DEFAULT_TABLE table=test-table [id=4c4e1c94cecb45c28e61f8a8811124b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:06.542073 23049 tablet_service.cc:1467] Processing CreateTablet for tablet 5780ebd3860c47398319ceb79dd568de (DEFAULT_TABLE table=test-table [id=4c4e1c94cecb45c28e61f8a8811124b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:06.542052 22544 tablet_service.cc:1467] Processing CreateTablet for tablet 5780ebd3860c47398319ceb79dd568de (DEFAULT_TABLE table=test-table [id=4c4e1c94cecb45c28e61f8a8811124b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:06.542450 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780ebd3860c47398319ceb79dd568de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:06.542452 22544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780ebd3860c47398319ceb79dd568de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:06.542452 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5780ebd3860c47398319ceb79dd568de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:06.546005 23231 tablet_bootstrap.cc:492] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Bootstrap starting.
I20260812 00:07:06.546427 23242 tablet_bootstrap.cc:492] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Bootstrap starting.
I20260812 00:07:06.546716 23241 tablet_bootstrap.cc:492] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Bootstrap starting.
I20260812 00:07:06.546829 23231 tablet_bootstrap.cc:654] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:06.547089 23242 tablet_bootstrap.cc:654] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:06.547195 23231 log.cc:824] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:06.547354 23241 tablet_bootstrap.cc:654] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:06.547436 23242 log.cc:824] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:06.547693 23241 log.cc:824] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:06.548096 23231 tablet_bootstrap.cc:492] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: No bootstrap required, opened a new log
I20260812 00:07:06.548180 23231 ts_tablet_manager.cc:1397] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:06.548411 23241 tablet_bootstrap.cc:492] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: No bootstrap required, opened a new log
I20260812 00:07:06.548465 23241 ts_tablet_manager.cc:1397] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:06.549753 23231 raft_consensus.cc:348] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.549922 23231 raft_consensus.cc:374] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:06.549971 23231 raft_consensus.cc:729] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374cc386b90d4ff1841e59f7c63635fc, State: Initialized, Role: FOLLOWER
I20260812 00:07:06.550173 23231 consensus_queue.cc:260] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.550165 23241 raft_consensus.cc:348] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.550354 23241 raft_consensus.cc:374] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:06.550402 23241 raft_consensus.cc:729] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Initialized, Role: FOLLOWER
I20260812 00:07:06.550493 23231 ts_tablet_manager.cc:1428] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:06.550525 22921 heartbeater.cc:503] Master 127.7.85.254:41095 was elected leader, sending a full tablet report...
I20260812 00:07:06.550554 23241 consensus_queue.cc:260] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.550856 23241 ts_tablet_manager.cc:1428] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 00:07:06.550896 23177 heartbeater.cc:503] Master 127.7.85.254:41095 was elected leader, sending a full tablet report...
I20260812 00:07:06.551235 23242 tablet_bootstrap.cc:492] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: No bootstrap required, opened a new log
I20260812 00:07:06.551308 23242 ts_tablet_manager.cc:1397] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 00:07:06.553296 23242 raft_consensus.cc:348] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.553448 23242 raft_consensus.cc:374] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:06.553470 23242 raft_consensus.cc:729] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cf7d229e9c4cb1ac292c639fa3ba58, State: Initialized, Role: FOLLOWER
I20260812 00:07:06.553580 23242 consensus_queue.cc:260] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.553835 23242 ts_tablet_manager.cc:1428] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 00:07:06.553984 22668 heartbeater.cc:503] Master 127.7.85.254:41095 was elected leader, sending a full tablet report...
W20260812 00:07:06.608299 22669 tablet.cc:2369] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:07:06.728642 23178 tablet.cc:2369] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:07:06.755987 23254 raft_consensus.cc:482] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:06.756148 23254 raft_consensus.cc:504] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.756609 23254 leader_election.cc:288] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719), f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853)
I20260812 00:07:06.759912 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.756870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33719 (local address 127.7.85.195:37399)
0812 00:07:06.756979 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.757007 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.757099 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.757340 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.757341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.757344 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:06.757434 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.757436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.758168 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.758169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.758985 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.758988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.759605 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.759606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.759732 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.759737 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.759797 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.759846 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 00:07:06.759974 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.757030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:37399 (local address 127.7.85.194:33719)
0812 00:07:06.757091 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.757093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.757105 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:07:06.757240 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.757242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.757260 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.757320 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:06.757533 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.758090 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.759069 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.759515 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.759576 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.759583 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.759841 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.759886 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 00:07:06.760138 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.757094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:39309 (local address 127.7.85.193:42853)
0812 00:07:06.757212 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.757215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.757321 (+ 106us) server_negotiation.cc:407] Connection header received
0812 00:07:06.757358 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.757359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.757371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.757439 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:06.757704 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.758237 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.759244 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.759719 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.759807 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.759815 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.760030 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.760080 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:07:06.760469 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.757007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:42853 (local address 127.7.85.195:39309)
0812 00:07:06.757222 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:06.757242 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:06.757310 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:06.757523 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:06.757524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:06.757526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:06.757616 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.757618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.758322 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.758323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.759143 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:06.759145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.759846 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.759847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:06.759923 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.759927 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:06.760320 (+ 393us) client_negotiation.cc:240] Negotiation successful
0812 00:07:06.760371 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260812 00:07:06.760408 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780ebd3860c47398319ceb79dd568de" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc" is_pre_election: true
I20260812 00:07:06.760587 22837 raft_consensus.cc:2454] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:06.760875 22959 leader_election.cc:302] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, 374cc386b90d4ff1841e59f7c63635fc; no voters:
I20260812 00:07:06.760901 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780ebd3860c47398319ceb79dd568de" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" is_pre_election: true
I20260812 00:07:06.761037 22581 raft_consensus.cc:2454] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:06.761060 23254 raft_consensus.cc:2781] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:06.761152 23254 raft_consensus.cc:482] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:06.761215 23254 raft_consensus.cc:3037] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:06.762260 23254 raft_consensus.cc:504] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.762383 23254 leader_election.cc:288] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Requested vote from peers 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719), f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853)
I20260812 00:07:06.762667 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780ebd3860c47398319ceb79dd568de" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
I20260812 00:07:06.762635 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780ebd3860c47398319ceb79dd568de" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc"
I20260812 00:07:06.762748 22581 raft_consensus.cc:3037] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:06.762768 22837 raft_consensus.cc:3037] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:06.763653 22581 raft_consensus.cc:2454] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:06.763746 22837 raft_consensus.cc:2454] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:06.763906 22959 leader_election.cc:302] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:06.764040 23254 raft_consensus.cc:2781] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:06.764196 23254 raft_consensus.cc:686] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Becoming Leader. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Running, Role: LEADER
I20260812 00:07:06.764344 23254 consensus_queue.cc:237] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } }
I20260812 00:07:06.765307 22298 catalog_manager.cc:5495] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 229509fd02a349618fe8eb51ead315b9 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } health_report { overall_health: UNKNOWN } } }
W20260812 00:07:06.795187 22922 tablet.cc:2369] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:07:06.826457 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:06.823659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.7.85.195:44807)
0812 00:07:06.823766 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:06.823768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:06.823776 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:06.823819 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:06.823820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:06.823842 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:06.823992 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:06.824293 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.824777 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.825632 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:06.825828 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:06.825905 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:06.826082 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:06.826199 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:06.826317 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:06.826334 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 00:07:06.826355 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:07:06.828480 22581 raft_consensus.cc:1261] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:06.828529 22837 raft_consensus.cc:1261] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:06.828930 23254 consensus_queue.cc:1035] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:06.829118 23260 consensus_queue.cc:1035] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:06.832810 23266 mvcc.cc:205] Tried to move back new op lower bound from 7317476257086304256 to 7317476256827338752. Current Snapshot: MvccSnapshot[applied={T|T < 7317476257086304256 or (T in {7317476257086304256})}]
I20260812 00:07:06.832942 23265 mvcc.cc:205] Tried to move back new op lower bound from 7317476257086304256 to 7317476256827338752. Current Snapshot: MvccSnapshot[applied={T|T < 7317476257086304256}]
I20260812 00:07:06.834815 22300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:06.834946 22300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:06.836210 22300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Sending DeleteTablet for 3 replicas of tablet 5780ebd3860c47398319ceb79dd568de
I20260812 00:07:06.836620 23049 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780ebd3860c47398319ceb79dd568de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:06 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:37116
I20260812 00:07:06.836700 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780ebd3860c47398319ceb79dd568de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:06 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39126
I20260812 00:07:06.836738 22544 tablet_service.cc:1514] Processing DeleteTablet for tablet 5780ebd3860c47398319ceb79dd568de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:06 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:33714
I20260812 00:07:06.837030 23285 tablet_replica.cc:330] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: stopping tablet replica
I20260812 00:07:06.837046 23286 tablet_replica.cc:330] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: stopping tablet replica
I20260812 00:07:06.837262 23283 tablet_replica.cc:330] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: stopping tablet replica
I20260812 00:07:06.837198 22300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:07:06.837369 22300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:06.837366 23283 raft_consensus.cc:2229] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:06.837471 23283 raft_consensus.cc:2258] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:06.837603 23285 raft_consensus.cc:2229] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:06.837716 23286 raft_consensus.cc:2229] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:06.837796 23285 raft_consensus.cc:2258] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:06.838413 23285 ts_tablet_manager.cc:1905] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:06.838413 23283 ts_tablet_manager.cc:1905] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:06.839761 22799 tablet_service.cc:1467] Processing CreateTablet for tablet c639506e9911403aab01a2eeb45f5fb6 (DEFAULT_TABLE table=test-table [id=eb600ee4c3f547409b56bad4afc48b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:06.839876 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c639506e9911403aab01a2eeb45f5fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:06.840446 23285 ts_tablet_manager.cc:1918] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:06.840464 23283 ts_tablet_manager.cc:1918] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:06.840523 23285 log.cc:1196] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/5780ebd3860c47398319ceb79dd568de
I20260812 00:07:06.840543 23283 log.cc:1196] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/5780ebd3860c47398319ceb79dd568de
I20260812 00:07:06.840822 23285 ts_tablet_manager.cc:1939] T 5780ebd3860c47398319ceb79dd568de P 374cc386b90d4ff1841e59f7c63635fc: Deleting consensus metadata
I20260812 00:07:06.840840 23283 ts_tablet_manager.cc:1939] T 5780ebd3860c47398319ceb79dd568de P 229509fd02a349618fe8eb51ead315b9: Deleting consensus metadata
I20260812 00:07:06.841383 22273 catalog_manager.cc:4841] TS 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807): tablet 5780ebd3860c47398319ceb79dd568de (table test-table [id=4c4e1c94cecb45c28e61f8a8811124b4]) successfully deleted
I20260812 00:07:06.841416 23049 tablet_service.cc:1467] Processing CreateTablet for tablet c639506e9911403aab01a2eeb45f5fb6 (DEFAULT_TABLE table=test-table [id=eb600ee4c3f547409b56bad4afc48b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:06.841517 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c639506e9911403aab01a2eeb45f5fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:06.842376 23241 tablet_bootstrap.cc:492] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Bootstrap starting.
I20260812 00:07:06.842415 23231 tablet_bootstrap.cc:492] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Bootstrap starting.
I20260812 00:07:06.842821 23231 tablet_bootstrap.cc:654] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:06.842821 23241 tablet_bootstrap.cc:654] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:06.843580 23241 tablet_bootstrap.cc:492] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: No bootstrap required, opened a new log
I20260812 00:07:06.843637 23241 ts_tablet_manager.cc:1397] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:06.843801 22266 catalog_manager.cc:4841] TS 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719): tablet 5780ebd3860c47398319ceb79dd568de (table test-table [id=4c4e1c94cecb45c28e61f8a8811124b4]) successfully deleted
I20260812 00:07:06.843792 23241 raft_consensus.cc:348] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:06.843871 23241 raft_consensus.cc:374] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:06.843892 23241 raft_consensus.cc:729] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Initialized, Role: FOLLOWER
I20260812 00:07:06.843947 23241 consensus_queue.cc:260] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:06.844051 23241 ts_tablet_manager.cc:1428] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.844794 23231 tablet_bootstrap.cc:492] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: No bootstrap required, opened a new log
I20260812 00:07:06.844861 23231 ts_tablet_manager.cc:1397] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:06.845013 23231 raft_consensus.cc:348] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:06.845093 23231 raft_consensus.cc:374] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:06.845127 23231 raft_consensus.cc:729] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374cc386b90d4ff1841e59f7c63635fc, State: Initialized, Role: FOLLOWER
I20260812 00:07:06.845196 23231 consensus_queue.cc:260] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:06.845309 23231 ts_tablet_manager.cc:1428] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:06.839636 22544 tablet_service.cc:1467] Processing CreateTablet for tablet c639506e9911403aab01a2eeb45f5fb6 (DEFAULT_TABLE table=test-table [id=eb600ee4c3f547409b56bad4afc48b37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:06.845588 22544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c639506e9911403aab01a2eeb45f5fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:06.846865 23242 tablet_bootstrap.cc:492] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Bootstrap starting.
I20260812 00:07:06.847270 23286 raft_consensus.cc:2258] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:06.847312 23242 tablet_bootstrap.cc:654] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:06.848028 23286 ts_tablet_manager.cc:1905] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:06.848172 23242 tablet_bootstrap.cc:492] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: No bootstrap required, opened a new log
I20260812 00:07:06.848224 23242 ts_tablet_manager.cc:1397] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:06.848389 23242 raft_consensus.cc:348] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:06.848469 23242 raft_consensus.cc:374] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:06.848491 23242 raft_consensus.cc:729] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cf7d229e9c4cb1ac292c639fa3ba58, State: Initialized, Role: FOLLOWER
I20260812 00:07:06.848536 23242 consensus_queue.cc:260] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:06.848645 23242 ts_tablet_manager.cc:1428] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:07:06.850024 23286 ts_tablet_manager.cc:1918] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:06.850091 23286 log.cc:1196] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/5780ebd3860c47398319ceb79dd568de
I20260812 00:07:06.850282 23286 ts_tablet_manager.cc:1939] T 5780ebd3860c47398319ceb79dd568de P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting consensus metadata
I20260812 00:07:06.851092 22266 catalog_manager.cc:4841] TS f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853): tablet 5780ebd3860c47398319ceb79dd568de (table test-table [id=4c4e1c94cecb45c28e61f8a8811124b4]) successfully deleted
I20260812 00:07:07.186028 23254 raft_consensus.cc:482] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:07.186163 23254 raft_consensus.cc:504] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.186381 23254 leader_election.cc:288] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:07.186859 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c639506e9911403aab01a2eeb45f5fb6" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" is_pre_election: true
I20260812 00:07:07.186971 22581 raft_consensus.cc:2454] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:07.186942 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c639506e9911403aab01a2eeb45f5fb6" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc" is_pre_election: true
I20260812 00:07:07.187121 22837 raft_consensus.cc:2454] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:07.187192 22959 leader_election.cc:302] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:07.187333 23254 raft_consensus.cc:2781] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:07.187402 23254 raft_consensus.cc:482] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:07.187429 23254 raft_consensus.cc:3037] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.188146 23254 raft_consensus.cc:504] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.188299 23254 leader_election.cc:288] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Requested vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:07.188699 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c639506e9911403aab01a2eeb45f5fb6" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc"
I20260812 00:07:07.188843 22837 raft_consensus.cc:3037] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.188827 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c639506e9911403aab01a2eeb45f5fb6" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
I20260812 00:07:07.189054 22581 raft_consensus.cc:3037] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.189678 22837 raft_consensus.cc:2454] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:07.189713 22581 raft_consensus.cc:2454] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:07.189909 22959 leader_election.cc:302] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, 374cc386b90d4ff1841e59f7c63635fc; no voters:
I20260812 00:07:07.190012 23254 raft_consensus.cc:2781] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:07.190071 23254 raft_consensus.cc:686] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Becoming Leader. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Running, Role: LEADER
I20260812 00:07:07.190122 23254 consensus_queue.cc:237] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.190785 22298 catalog_manager.cc:5495] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 229509fd02a349618fe8eb51ead315b9 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } health_report { overall_health: HEALTHY } } }
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.254@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:07.230486 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.222021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41095 (local address 127.0.0.1:50460)
0812 00:07:07.222211 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.222230 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.222307 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.222514 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.222516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.222857 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:07.222908 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.222910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.223675 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.223679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.224213 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.224214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.224652 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.224654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.224719 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.225399 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:07.225404 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:07.226969 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:07.228809 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:07.228820 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:07.228823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:07.229045 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:07.229435 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:07.229438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:07.229439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:07.229634 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:07.230142 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:07.230150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:07.230210 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.230296 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.230402 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260812 00:07:07.230607 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.222076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.7.85.254:41095)
0812 00:07:07.222244 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.222246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.222357 (+ 111us) server_negotiation.cc:407] Connection header received
0812 00:07:07.222389 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.222390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.222402 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.222478 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:07.223012 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.223577 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.224338 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.224546 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.224605 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.227066 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:07.227067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:07.227067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:07.227074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:07.228703 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:07.229139 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:07.229141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:07.229141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:07.229300 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:07.229785 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:07.229787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:07.229788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:07.230005 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:07.230115 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.230394 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.230493 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260812 00:07:07.233227 22298 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:50460:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 00:07:07.233450 22298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:07.235118 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 809bcb76afa04ce5b59c70fd6d7dbdcb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=da607cb1842a4e3f8adbc37c467574a1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:07:07.235105 22544 tablet_service.cc:1467] Processing CreateTablet for tablet 809bcb76afa04ce5b59c70fd6d7dbdcb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=da607cb1842a4e3f8adbc37c467574a1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:07:07.235108 23049 tablet_service.cc:1467] Processing CreateTablet for tablet 809bcb76afa04ce5b59c70fd6d7dbdcb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=da607cb1842a4e3f8adbc37c467574a1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:07:07.235307 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809bcb76afa04ce5b59c70fd6d7dbdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.235307 22544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809bcb76afa04ce5b59c70fd6d7dbdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.235316 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809bcb76afa04ce5b59c70fd6d7dbdcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.236640 23231 tablet_bootstrap.cc:492] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: Bootstrap starting.
I20260812 00:07:07.236960 23241 tablet_bootstrap.cc:492] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: Bootstrap starting.
I20260812 00:07:07.236967 23242 tablet_bootstrap.cc:492] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: Bootstrap starting.
I20260812 00:07:07.237157 23231 tablet_bootstrap.cc:654] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.237362 23241 tablet_bootstrap.cc:654] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.237453 23242 tablet_bootstrap.cc:654] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.237993 23241 tablet_bootstrap.cc:492] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: No bootstrap required, opened a new log
I20260812 00:07:07.238062 23241 ts_tablet_manager.cc:1397] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:07.238188 23241 raft_consensus.cc:348] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.238273 23241 raft_consensus.cc:374] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.238270 23231 tablet_bootstrap.cc:492] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: No bootstrap required, opened a new log
I20260812 00:07:07.238297 23241 raft_consensus.cc:729] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.238313 23231 ts_tablet_manager.cc:1397] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:07:07.238351 23241 consensus_queue.cc:260] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.238461 23241 ts_tablet_manager.cc:1428] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:07.238477 23231 raft_consensus.cc:348] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.238590 23231 raft_consensus.cc:374] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.238610 23231 raft_consensus.cc:729] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374cc386b90d4ff1841e59f7c63635fc, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.238485 23254 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.238715 23231 consensus_queue.cc:260] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.238749 23254 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.238837 23231 ts_tablet_manager.cc:1428] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:07.238878 23250 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.238965 23250 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.239006 23242 tablet_bootstrap.cc:492] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: No bootstrap required, opened a new log
I20260812 00:07:07.239076 23242 ts_tablet_manager.cc:1397] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:07.239369 23242 raft_consensus.cc:348] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.239465 23242 raft_consensus.cc:374] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.239499 23242 raft_consensus.cc:729] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cf7d229e9c4cb1ac292c639fa3ba58, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.239562 23242 consensus_queue.cc:260] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.239670 23242 ts_tablet_manager.cc:1428] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:07:07.239712 23255 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.239814 23255 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.252749 23254 raft_consensus.cc:482] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:07.252866 23254 raft_consensus.cc:504] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.253015 23254 leader_election.cc:288] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:07.253288 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809bcb76afa04ce5b59c70fd6d7dbdcb" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" is_pre_election: true
I20260812 00:07:07.253362 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809bcb76afa04ce5b59c70fd6d7dbdcb" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc" is_pre_election: true
I20260812 00:07:07.253429 22581 raft_consensus.cc:2454] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:07.253492 22837 raft_consensus.cc:2454] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:07.253638 22959 leader_election.cc:302] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, 374cc386b90d4ff1841e59f7c63635fc; no voters:
I20260812 00:07:07.253769 23254 raft_consensus.cc:2781] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:07.253805 23254 raft_consensus.cc:482] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:07.253837 23254 raft_consensus.cc:3037] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.254573 23254 raft_consensus.cc:504] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.254703 23254 leader_election.cc:288] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Requested vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:07.254908 22581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809bcb76afa04ce5b59c70fd6d7dbdcb" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
I20260812 00:07:07.254920 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809bcb76afa04ce5b59c70fd6d7dbdcb" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc"
I20260812 00:07:07.255004 22581 raft_consensus.cc:3037] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.255020 22837 raft_consensus.cc:3037] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.255715 22581 raft_consensus.cc:2454] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:07.255827 22837 raft_consensus.cc:2454] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:07.255909 22959 leader_election.cc:302] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:07.256022 23254 raft_consensus.cc:2781] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:07.256094 23254 raft_consensus.cc:686] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Becoming Leader. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Running, Role: LEADER
I20260812 00:07:07.256146 23254 consensus_queue.cc:237] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.256423 23260 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: TxnStatusTablet state changed. Reason: New leader 229509fd02a349618fe8eb51ead315b9. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.256526 23260 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:07:07.256681 23320 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 00:07:07.256757 22300 catalog_manager.cc:5495] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 229509fd02a349618fe8eb51ead315b9 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } health_report { overall_health: HEALTHY } } }
I20260812 00:07:07.256862 23320 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 00:07:07.265558 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.262686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44807 (local address 127.0.0.1:37142)
0812 00:07:07.262772 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.262787 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.262844 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.263005 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.263006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.263007 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:07:07.263063 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.263065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.263857 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.263859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.264655 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.264658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.265091 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.265092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.265150 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.265191 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.265375 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.265382 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.265435 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.265487 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 00:07:07.265542 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.262775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.7.85.195:44807)
0812 00:07:07.262831 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.262833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.262866 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:07:07.262922 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.262923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.262936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.262997 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:07.263132 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.263655 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.264764 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.265012 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.265070 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.265264 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.265303 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.265376 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.265439 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.265462 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 00:07:07.266587 22581 raft_consensus.cc:1261] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.266583 22837 raft_consensus.cc:1261] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.266845 23254 consensus_queue.cc:1035] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.266990 23260 consensus_queue.cc:1035] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.267513 23255 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: TxnStatusTablet state changed. Reason: New leader 229509fd02a349618fe8eb51ead315b9. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.267613 23255 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.267899 23250 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: TxnStatusTablet state changed. Reason: New leader 229509fd02a349618fe8eb51ead315b9. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.267994 23250 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.268407 23263 mvcc.cc:205] Tried to move back new op lower bound from 7317476258882142208 to 7317476258841489408. Current Snapshot: MvccSnapshot[applied={T|T < 7317476258882142208}]
I20260812 00:07:07.268482 23260 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.268553 23260 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:07:07.268708 23255 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.268739 23254 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.268787 23255 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P f8cf7d229e9c4cb1ac292c639fa3ba58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.268801 23254 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 229509fd02a349618fe8eb51ead315b9: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:07:07.269109 23250 tablet_replica.cc:439] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } } }
I20260812 00:07:07.269212 23250 tablet_replica.cc:442] T 809bcb76afa04ce5b59c70fd6d7dbdcb P 374cc386b90d4ff1841e59f7c63635fc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:07:07.274789 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.272233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:52293 (local address 127.7.85.195:44807)
0812 00:07:07.272274 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.272276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.272341 (+ 65us) server_negotiation.cc:407] Connection header received
0812 00:07:07.272411 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.272412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.272420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.272466 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:07.272580 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.273021 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.274125 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.274304 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.274362 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.274536 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.274563 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.274602 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.274678 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.274703 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":19}
I20260812 00:07:07.274852 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.272127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44807 (local address 127.7.85.195:52293)
0812 00:07:07.272288 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.272299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.272356 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.272478 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.272478 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.272480 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:07:07.272527 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.272528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.273089 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.273090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.274030 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.274033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.274381 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.274382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.274440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.274455 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.274621 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.274623 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.274677 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.274723 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260812 00:07:07.276854 22837 raft_consensus.cc:1261] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.276885 22580 raft_consensus.cc:1261] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.277062 23260 consensus_queue.cc:1035] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.277163 23254 consensus_queue.cc:1035] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.292280 22298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:07.292392 22298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:07.293133 22298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Sending DeleteTablet for 3 replicas of tablet c639506e9911403aab01a2eeb45f5fb6
I20260812 00:07:07.293478 22544 tablet_service.cc:1514] Processing DeleteTablet for tablet c639506e9911403aab01a2eeb45f5fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:33714
I20260812 00:07:07.293497 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet c639506e9911403aab01a2eeb45f5fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39126
I20260812 00:07:07.293589 23286 tablet_replica.cc:330] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: stopping tablet replica
I20260812 00:07:07.293623 23285 tablet_replica.cc:330] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: stopping tablet replica
I20260812 00:07:07.293640 23286 raft_consensus.cc:2229] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:07.293665 23285 raft_consensus.cc:2229] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:07.293678 23286 raft_consensus.cc:2258] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.293700 23285 raft_consensus.cc:2258] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.293913 23049 tablet_service.cc:1514] Processing DeleteTablet for tablet c639506e9911403aab01a2eeb45f5fb6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:37116
I20260812 00:07:07.294001 23283 tablet_replica.cc:330] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: stopping tablet replica
I20260812 00:07:07.294050 23283 raft_consensus.cc:2229] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:07.294104 23283 raft_consensus.cc:2258] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.294431 23285 ts_tablet_manager.cc:1905] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.294529 23283 ts_tablet_manager.cc:1905] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.296373 23285 ts_tablet_manager.cc:1918] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:07:07.296432 23285 log.cc:1196] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/c639506e9911403aab01a2eeb45f5fb6
I20260812 00:07:07.296649 23285 ts_tablet_manager.cc:1939] T c639506e9911403aab01a2eeb45f5fb6 P 374cc386b90d4ff1841e59f7c63635fc: Deleting consensus metadata
I20260812 00:07:07.297072 22266 catalog_manager.cc:4841] TS 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719): tablet c639506e9911403aab01a2eeb45f5fb6 (table test-table [id=eb600ee4c3f547409b56bad4afc48b37]) successfully deleted
I20260812 00:07:07.300060 23286 ts_tablet_manager.cc:1905] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.300729 23283 ts_tablet_manager.cc:1918] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:07:07.300861 23283 log.cc:1196] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/c639506e9911403aab01a2eeb45f5fb6
I20260812 00:07:07.301213 23283 ts_tablet_manager.cc:1939] T c639506e9911403aab01a2eeb45f5fb6 P 229509fd02a349618fe8eb51ead315b9: Deleting consensus metadata
I20260812 00:07:07.301823 22273 catalog_manager.cc:4841] TS 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807): tablet c639506e9911403aab01a2eeb45f5fb6 (table test-table [id=eb600ee4c3f547409b56bad4afc48b37]) successfully deleted
I20260812 00:07:07.303125 23286 ts_tablet_manager.cc:1918] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:07:07.303289 23286 log.cc:1196] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/c639506e9911403aab01a2eeb45f5fb6
I20260812 00:07:07.303524 23286 ts_tablet_manager.cc:1939] T c639506e9911403aab01a2eeb45f5fb6 P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting consensus metadata
I20260812 00:07:07.304170 22266 catalog_manager.cc:4841] TS f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853): tablet c639506e9911403aab01a2eeb45f5fb6 (table test-table [id=eb600ee4c3f547409b56bad4afc48b37]) successfully deleted
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[22211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 00:07:07.328666 23315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:07.321992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.7.85.254:41095)
0812 00:07:07.322052 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.322055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.322230 (+ 175us) server_negotiation.cc:407] Connection header received
0812 00:07:07.322264 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.322265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.322281 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.322330 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:07.323081 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.323847 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.324957 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.325159 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.325315 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.328591 (+ 3276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50474: BlockingRecv error: recv got EOF from 127.0.0.1:50474 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[22211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM
I20260812 00:07:07.338145 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.329764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50478 (local address 127.7.85.254:41095)
0812 00:07:07.329806 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.329807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.329841 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:07:07.329916 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.329916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.329924 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.329971 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:07.330583 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.331070 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.331957 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.332155 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.332237 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.334897 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:07.334899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:07.334899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:07.334906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:07.336664 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:07.337070 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:07.337071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:07.337071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:07.337141 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:07.337630 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:07.337630 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:07.337631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:07.337762 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:07.337850 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.338003 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.338074 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:07:07.340912 22298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:07:07.341075 22298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:07.342687 22544 tablet_service.cc:1467] Processing CreateTablet for tablet 958e685ed2e54b2fb37490bfa4ff8036 (DEFAULT_TABLE table=test-table [id=cea28882908e419c844fd399d43ca1c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:07.342708 23049 tablet_service.cc:1467] Processing CreateTablet for tablet 958e685ed2e54b2fb37490bfa4ff8036 (DEFAULT_TABLE table=test-table [id=cea28882908e419c844fd399d43ca1c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:07.342751 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 958e685ed2e54b2fb37490bfa4ff8036 (DEFAULT_TABLE table=test-table [id=cea28882908e419c844fd399d43ca1c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:07.342859 22544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958e685ed2e54b2fb37490bfa4ff8036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.342859 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958e685ed2e54b2fb37490bfa4ff8036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.342864 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 958e685ed2e54b2fb37490bfa4ff8036. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.344230 23242 tablet_bootstrap.cc:492] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Bootstrap starting.
I20260812 00:07:07.344295 23241 tablet_bootstrap.cc:492] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Bootstrap starting.
I20260812 00:07:07.344663 23241 tablet_bootstrap.cc:654] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.344674 23242 tablet_bootstrap.cc:654] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.344951 23231 tablet_bootstrap.cc:492] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Bootstrap starting.
I20260812 00:07:07.345410 23231 tablet_bootstrap.cc:654] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.346001 23241 tablet_bootstrap.cc:492] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: No bootstrap required, opened a new log
I20260812 00:07:07.346047 23241 ts_tablet_manager.cc:1397] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:07.346136 23241 raft_consensus.cc:348] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.346199 23241 raft_consensus.cc:374] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.346212 23241 raft_consensus.cc:729] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.346272 23241 consensus_queue.cc:260] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.346375 23241 ts_tablet_manager.cc:1428] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:07.346850 23231 tablet_bootstrap.cc:492] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: No bootstrap required, opened a new log
I20260812 00:07:07.346921 23231 ts_tablet_manager.cc:1397] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:07:07.347081 23231 raft_consensus.cc:348] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.347131 23231 raft_consensus.cc:374] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.347146 23231 raft_consensus.cc:729] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374cc386b90d4ff1841e59f7c63635fc, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.347201 23231 consensus_queue.cc:260] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.347294 23231 ts_tablet_manager.cc:1428] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:07.348273 23242 tablet_bootstrap.cc:492] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: No bootstrap required, opened a new log
I20260812 00:07:07.348345 23242 ts_tablet_manager.cc:1397] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 00:07:07.348524 23242 raft_consensus.cc:348] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.348625 23242 raft_consensus.cc:374] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.348649 23242 raft_consensus.cc:729] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cf7d229e9c4cb1ac292c639fa3ba58, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.348701 23242 consensus_queue.cc:260] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.348819 23242 ts_tablet_manager.cc:1428] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:07:07.406013 23254 raft_consensus.cc:482] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:07.406186 23254 raft_consensus.cc:504] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.406502 23254 leader_election.cc:288] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:07.406711 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958e685ed2e54b2fb37490bfa4ff8036" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" is_pre_election: true
I20260812 00:07:07.406833 22580 raft_consensus.cc:2454] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:07.406796 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958e685ed2e54b2fb37490bfa4ff8036" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc" is_pre_election: true
I20260812 00:07:07.406913 22837 raft_consensus.cc:2454] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 0.
I20260812 00:07:07.406994 22959 leader_election.cc:302] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:07.407145 23254 raft_consensus.cc:2781] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:07.407209 23254 raft_consensus.cc:482] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:07.407244 23254 raft_consensus.cc:3037] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.407964 23254 raft_consensus.cc:504] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.408111 23254 leader_election.cc:288] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Requested vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719)
I20260812 00:07:07.408322 22837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958e685ed2e54b2fb37490bfa4ff8036" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374cc386b90d4ff1841e59f7c63635fc"
I20260812 00:07:07.408326 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "958e685ed2e54b2fb37490bfa4ff8036" candidate_uuid: "229509fd02a349618fe8eb51ead315b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
I20260812 00:07:07.408411 22837 raft_consensus.cc:3037] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.408432 22580 raft_consensus.cc:3037] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.409123 22580 raft_consensus.cc:2454] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:07.409224 22837 raft_consensus.cc:2454] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 229509fd02a349618fe8eb51ead315b9 in term 1.
I20260812 00:07:07.409593 22959 leader_election.cc:302] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 229509fd02a349618fe8eb51ead315b9, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:07.409796 23254 raft_consensus.cc:2781] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:07.409874 23254 raft_consensus.cc:686] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Becoming Leader. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Running, Role: LEADER
I20260812 00:07:07.409920 23254 consensus_queue.cc:237] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.410622 22298 catalog_manager.cc:5495] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 229509fd02a349618fe8eb51ead315b9 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "229509fd02a349618fe8eb51ead315b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } health_report { overall_health: HEALTHY } } }
I20260812 00:07:07.426703 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.423382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.7.85.195:44807)
0812 00:07:07.423450 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.423452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.423462 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:07.423496 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.423496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.423512 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.423570 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:07.424145 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.424652 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.425740 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.426111 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.426187 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.426382 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.426411 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.426454 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.426602 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.426627 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 00:07:07.428174 22837 raft_consensus.cc:1261] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.428352 22580 raft_consensus.cc:1261] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Refusing update from remote peer 229509fd02a349618fe8eb51ead315b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.428601 23254 consensus_queue.cc:1035] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.428768 23260 consensus_queue.cc:1035] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.432098 22294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:07.432299 22294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:07.433140 22294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Sending DeleteTablet for 3 replicas of tablet 958e685ed2e54b2fb37490bfa4ff8036
I20260812 00:07:07.433506 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet 958e685ed2e54b2fb37490bfa4ff8036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39126
I20260812 00:07:07.433562 23049 tablet_service.cc:1514] Processing DeleteTablet for tablet 958e685ed2e54b2fb37490bfa4ff8036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:37116
I20260812 00:07:07.433617 23285 tablet_replica.cc:330] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: stopping tablet replica
I20260812 00:07:07.433665 23285 raft_consensus.cc:2229] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:07.433719 23285 raft_consensus.cc:2258] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.433696 22544 tablet_service.cc:1514] Processing DeleteTablet for tablet 958e685ed2e54b2fb37490bfa4ff8036 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:33714
I20260812 00:07:07.433818 23283 tablet_replica.cc:330] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: stopping tablet replica
I20260812 00:07:07.433884 23283 raft_consensus.cc:2229] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:07.433836 22294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 00:07:07.433961 23283 raft_consensus.cc:2258] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 00:07:07.433977 22294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:07.434046 23286 tablet_replica.cc:330] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: stopping tablet replica
I20260812 00:07:07.434085 23285 ts_tablet_manager.cc:1905] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.434103 23286 raft_consensus.cc:2229] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:07.434147 23286 raft_consensus.cc:2258] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.434525 23286 ts_tablet_manager.cc:1905] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.435272 23049 tablet_service.cc:1467] Processing CreateTablet for tablet 0e6fb7dec53849edb19c8735492bc3cc (DEFAULT_TABLE table=test-table [id=5a400f553b3e455a82ce53cb85e67b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:07.435407 23049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e6fb7dec53849edb19c8735492bc3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.436514 23241 tablet_bootstrap.cc:492] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Bootstrap starting.
I20260812 00:07:07.436642 23285 ts_tablet_manager.cc:1918] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:07.436667 22544 tablet_service.cc:1467] Processing CreateTablet for tablet 0e6fb7dec53849edb19c8735492bc3cc (DEFAULT_TABLE table=test-table [id=5a400f553b3e455a82ce53cb85e67b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:07.436708 23285 log.cc:1196] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/958e685ed2e54b2fb37490bfa4ff8036
I20260812 00:07:07.436839 23286 ts_tablet_manager.cc:1918] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:07.436877 23241 tablet_bootstrap.cc:654] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.436887 23286 log.cc:1196] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/958e685ed2e54b2fb37490bfa4ff8036
I20260812 00:07:07.436962 23285 ts_tablet_manager.cc:1939] T 958e685ed2e54b2fb37490bfa4ff8036 P 374cc386b90d4ff1841e59f7c63635fc: Deleting consensus metadata
I20260812 00:07:07.437110 23286 ts_tablet_manager.cc:1939] T 958e685ed2e54b2fb37490bfa4ff8036 P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting consensus metadata
I20260812 00:07:07.437201 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 0e6fb7dec53849edb19c8735492bc3cc (DEFAULT_TABLE table=test-table [id=5a400f553b3e455a82ce53cb85e67b1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:07.437299 22544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e6fb7dec53849edb19c8735492bc3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.437316 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e6fb7dec53849edb19c8735492bc3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:07.437572 22266 catalog_manager.cc:4841] TS 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719): tablet 958e685ed2e54b2fb37490bfa4ff8036 (table test-table [id=cea28882908e419c844fd399d43ca1c4]) successfully deleted
I20260812 00:07:07.438333 23242 tablet_bootstrap.cc:492] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: Bootstrap starting.
I20260812 00:07:07.438405 23231 tablet_bootstrap.cc:492] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Bootstrap starting.
I20260812 00:07:07.438684 23283 ts_tablet_manager.cc:1905] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.438964 23231 tablet_bootstrap.cc:654] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.439018 23242 tablet_bootstrap.cc:654] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:07.439713 23241 tablet_bootstrap.cc:492] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: No bootstrap required, opened a new log
I20260812 00:07:07.439749 23242 tablet_bootstrap.cc:492] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: No bootstrap required, opened a new log
I20260812 00:07:07.439787 23242 ts_tablet_manager.cc:1397] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:07.439787 23241 ts_tablet_manager.cc:1397] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 00:07:07.439910 23241 raft_consensus.cc:348] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.439939 23242 raft_consensus.cc:348] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.439993 23241 raft_consensus.cc:374] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.440001 23242 raft_consensus.cc:374] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.440021 23241 raft_consensus.cc:729] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229509fd02a349618fe8eb51ead315b9, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.440022 23242 raft_consensus.cc:729] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8cf7d229e9c4cb1ac292c639fa3ba58, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.440073 23241 consensus_queue.cc:260] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.440073 23242 consensus_queue.cc:260] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.440163 23241 ts_tablet_manager.cc:1428] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:07.440171 23242 ts_tablet_manager.cc:1428] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:07.440521 23231 tablet_bootstrap.cc:492] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: No bootstrap required, opened a new log
I20260812 00:07:07.440574 23231 ts_tablet_manager.cc:1397] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:07:07.440724 22266 catalog_manager.cc:4841] TS f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853): tablet 958e685ed2e54b2fb37490bfa4ff8036 (table test-table [id=cea28882908e419c844fd399d43ca1c4]) successfully deleted
I20260812 00:07:07.440739 23231 raft_consensus.cc:348] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.440842 23231 raft_consensus.cc:374] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:07.440871 23231 raft_consensus.cc:729] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374cc386b90d4ff1841e59f7c63635fc, State: Initialized, Role: FOLLOWER
I20260812 00:07:07.440928 23231 consensus_queue.cc:260] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.440987 23283 ts_tablet_manager.cc:1918] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:07.441026 23231 ts_tablet_manager.cc:1428] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 00:07:07.441051 23283 log.cc:1196] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/958e685ed2e54b2fb37490bfa4ff8036
I20260812 00:07:07.441318 23283 ts_tablet_manager.cc:1939] T 958e685ed2e54b2fb37490bfa4ff8036 P 229509fd02a349618fe8eb51ead315b9: Deleting consensus metadata
I20260812 00:07:07.441691 22273 catalog_manager.cc:4841] TS 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807): tablet 958e685ed2e54b2fb37490bfa4ff8036 (table test-table [id=cea28882908e419c844fd399d43ca1c4]) successfully deleted
I20260812 00:07:07.486500 23250 raft_consensus.cc:482] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:07.486658 23250 raft_consensus.cc:504] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.487093 23250 leader_election.cc:288] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807)
I20260812 00:07:07.491474 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.487296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:42853 (local address 127.7.85.194:54287)
0812 00:07:07.487475 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.487484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.487527 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.487711 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.487712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.487714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:07.487828 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.487832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.488848 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.488848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.490291 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.490295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.491142 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.491143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.491249 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.491253 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.491308 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.491335 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":100,"thread_start_us":106,"threads_started":1}
I20260812 00:07:07.491515 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.487348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:50401 (local address 127.7.85.195:44807)
0812 00:07:07.487404 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.487406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.487536 (+ 130us) server_negotiation.cc:407] Connection header received
0812 00:07:07.487583 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.487585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.487598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.487669 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:07.487937 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.488700 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.490411 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.490990 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.491081 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.491095 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.491394 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.491439 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:07:07.491531 23392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.487426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:54287 (local address 127.7.85.193:42853)
0812 00:07:07.487573 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.487576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.487584 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:07.487612 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.487613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.487623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.487688 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:07.487940 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.488717 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.490411 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.491005 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.491096 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.491106 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.491394 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.491439 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260812 00:07:07.491473 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.487276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44807 (local address 127.7.85.194:50401)
0812 00:07:07.487441 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.487453 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.487526 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.487711 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.487712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.487716 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:07.487828 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.487830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.488846 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.488848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.490291 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.490295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.491127 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.491129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.491209 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.491213 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.491283 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.491330 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":113,"thread_start_us":87,"threads_started":1}
I20260812 00:07:07.491804 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6fb7dec53849edb19c8735492bc3cc" candidate_uuid: "374cc386b90d4ff1841e59f7c63635fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" is_pre_election: true
I20260812 00:07:07.491915 22580 raft_consensus.cc:2454] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 374cc386b90d4ff1841e59f7c63635fc in term 0.
I20260812 00:07:07.491906 23088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6fb7dec53849edb19c8735492bc3cc" candidate_uuid: "374cc386b90d4ff1841e59f7c63635fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229509fd02a349618fe8eb51ead315b9" is_pre_election: true
I20260812 00:07:07.492067 23088 raft_consensus.cc:2454] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 374cc386b90d4ff1841e59f7c63635fc in term 0.
I20260812 00:07:07.492110 22712 leader_election.cc:302] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 374cc386b90d4ff1841e59f7c63635fc, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:07.492260 23250 raft_consensus.cc:2781] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:07.492321 23250 raft_consensus.cc:482] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:07.492362 23250 raft_consensus.cc:3037] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.493101 23250 raft_consensus.cc:504] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.493321 23250 leader_election.cc:288] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [CANDIDATE]: Term 1 election: Requested vote from peers f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853), 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807)
I20260812 00:07:07.493494 23088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6fb7dec53849edb19c8735492bc3cc" candidate_uuid: "374cc386b90d4ff1841e59f7c63635fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "229509fd02a349618fe8eb51ead315b9"
I20260812 00:07:07.493602 23088 raft_consensus.cc:3037] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.493655 22580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e6fb7dec53849edb19c8735492bc3cc" candidate_uuid: "374cc386b90d4ff1841e59f7c63635fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58"
I20260812 00:07:07.493760 22580 raft_consensus.cc:3037] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:07.494411 22580 raft_consensus.cc:2454] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374cc386b90d4ff1841e59f7c63635fc in term 1.
I20260812 00:07:07.494445 23088 raft_consensus.cc:2454] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374cc386b90d4ff1841e59f7c63635fc in term 1.
I20260812 00:07:07.494712 22712 leader_election.cc:302] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 374cc386b90d4ff1841e59f7c63635fc, f8cf7d229e9c4cb1ac292c639fa3ba58; no voters:
I20260812 00:07:07.494891 23250 raft_consensus.cc:2781] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:07.495121 23250 raft_consensus.cc:686] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 1 LEADER]: Becoming Leader. State: Replica: 374cc386b90d4ff1841e59f7c63635fc, State: Running, Role: LEADER
I20260812 00:07:07.495218 23250 consensus_queue.cc:237] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } }
I20260812 00:07:07.496513 22298 catalog_manager.cc:5495] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194). New cstate: current_term: 1 leader_uuid: "374cc386b90d4ff1841e59f7c63635fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374cc386b90d4ff1841e59f7c63635fc" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33719 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 } health_report { overall_health: UNKNOWN } } }
I20260812 00:07:07.506381 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.502994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39134 (local address 127.7.85.194:33719)
0812 00:07:07.503182 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.503183 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.503189 (+ 6us) server_negotiation.cc:407] Connection header received
0812 00:07:07.503222 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.503223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.503247 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.503322 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:07.503545 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.504074 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.504996 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.505469 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.505538 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.505768 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.505909 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.505983 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.506211 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.506236 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":131,"threads_started":1}
I20260812 00:07:07.513130 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.510105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44807 (local address 127.7.85.194:50353)
0812 00:07:07.510285 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.510297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.510359 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.510485 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.510486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.510488 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:07:07.510555 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.510556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.511265 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.511266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.511940 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.511943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.512514 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.512515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.512588 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.512605 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.512932 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.512935 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.512994 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.513045 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260812 00:07:07.513258 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.510251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:50353 (local address 127.7.85.195:44807)
0812 00:07:07.510323 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.510325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.510346 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:07:07.510418 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.510419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.510429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.510472 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:07.510658 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.511151 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.512114 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.512372 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.512477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.512758 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.512796 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.512885 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.513061 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.513091 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 00:07:07.515975 22580 raft_consensus.cc:1261] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Refusing update from remote peer 374cc386b90d4ff1841e59f7c63635fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.516019 23088 raft_consensus.cc:1261] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Refusing update from remote peer 374cc386b90d4ff1841e59f7c63635fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:07.516391 23250 consensus_queue.cc:1035] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "229509fd02a349618fe8eb51ead315b9" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.516551 23393 consensus_queue.cc:1035] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8cf7d229e9c4cb1ac292c639fa3ba58" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 42853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:07.525468 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.522920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:56901 (local address 127.7.85.194:33719)
0812 00:07:07.522965 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:07.522967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:07.522975 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:07.522997 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:07.522998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:07.523007 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:07.523087 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:07.523261 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.523745 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.524577 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.524904 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.524998 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.525200 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.525242 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.525303 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:07.525371 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 00:07:07.525399 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:07:07.525475 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.522713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33719 (local address 127.7.85.195:56901)
0812 00:07:07.522789 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.522802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:07.522843 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:07.523090 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:07.523091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:07.523093 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:07:07.523146 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.523148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.523841 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.523842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.524456 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:07.524459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:07.525017 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:07.525018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:07.525084 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:07.525091 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:07:07.525315 (+ 224us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:07:07.525318 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:07:07.525367 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 00:07:07.525415 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 00:07:07.536242 22300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:07.536384 22300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:07.537148 22300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e410903f9e4efab67c0af1bf3c1ac9: Sending DeleteTablet for 3 replicas of tablet 0e6fb7dec53849edb19c8735492bc3cc
I20260812 00:07:07.537516 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e6fb7dec53849edb19c8735492bc3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39126
I20260812 00:07:07.537546 22544 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e6fb7dec53849edb19c8735492bc3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:33714
I20260812 00:07:07.537618 23285 tablet_replica.cc:330] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: stopping tablet replica
I20260812 00:07:07.537632 23286 tablet_replica.cc:330] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: stopping tablet replica
I20260812 00:07:07.537665 23285 raft_consensus.cc:2229] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:07.537683 23286 raft_consensus.cc:2229] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:07.537719 23286 raft_consensus.cc:2258] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.537672 23049 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e6fb7dec53849edb19c8735492bc3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:07 UTC) from {username='oryx', principal='oryx/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:37116
I20260812 00:07:07.537766 23285 raft_consensus.cc:2258] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.537856 23283 tablet_replica.cc:330] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: stopping tablet replica
I20260812 00:07:07.537914 23283 raft_consensus.cc:2229] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:07.537950 23283 raft_consensus.cc:2258] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:07.538043 23286 ts_tablet_manager.cc:1905] T 0e6fb7dec53849edb19c8735492bc3cc P f8cf7d229e9c4cb1ac292c639fa3ba58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.538251 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 22423
I20260812 00:07:07.549805 23285 ts_tablet_manager.cc:1905] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.551472 23283 ts_tablet_manager.cc:1905] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:07.552000 23285 ts_tablet_manager.cc:1918] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:07:07.552083 23285 log.cc:1196] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/0e6fb7dec53849edb19c8735492bc3cc
I20260812 00:07:07.552434 23285 ts_tablet_manager.cc:1939] T 0e6fb7dec53849edb19c8735492bc3cc P 374cc386b90d4ff1841e59f7c63635fc: Deleting consensus metadata
I20260812 00:07:07.553229 22266 catalog_manager.cc:4841] TS 374cc386b90d4ff1841e59f7c63635fc (127.7.85.194:33719): tablet 0e6fb7dec53849edb19c8735492bc3cc (table test-table [id=5a400f553b3e455a82ce53cb85e67b1d]) successfully deleted
I20260812 00:07:07.553299 23283 ts_tablet_manager.cc:1918] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:07:07.553356 23283 log.cc:1196] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/0e6fb7dec53849edb19c8735492bc3cc
I20260812 00:07:07.553622 23283 ts_tablet_manager.cc:1939] T 0e6fb7dec53849edb19c8735492bc3cc P 229509fd02a349618fe8eb51ead315b9: Deleting consensus metadata
I20260812 00:07:07.554082 22273 catalog_manager.cc:4841] TS 229509fd02a349618fe8eb51ead315b9 (127.7.85.195:44807): tablet 0e6fb7dec53849edb19c8735492bc3cc (table test-table [id=5a400f553b3e455a82ce53cb85e67b1d]) successfully deleted
W20260812 00:07:07.569767 22266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:42853 (error 108)
I20260812 00:07:07.570216 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 22684
I20260812 00:07:07.580075 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:07.570306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:42853 (local address 127.0.0.1:33730)
0812 00:07:07.579906 (+ 9600us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:07.580004 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:42853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9565,"thread_start_us":68,"threads_started":1}
W20260812 00:07:07.581979 22266 catalog_manager.cc:4584] TS f8cf7d229e9c4cb1ac292c639fa3ba58 (127.7.85.193:42853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e6fb7dec53849edb19c8735492bc3cc: Network error: Client connection negotiation failed: client connection to 127.7.85.193:42853: connect: Connection refused (error 111)
I20260812 00:07:07.593945 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 22932
I20260812 00:07:07.614620 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 22236
2026-08-12T00:07:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3385 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[23425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[23425](info): set up 2 sockets
Aug 12 00:07:07 dist-test-slave-z8x9 krb5kdc[23425](info): commencing operation
krb5kdc: starting...
W20260812 00:07:08.982941 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:07:09 dist-test-slave-z8x9 krb5kdc[23425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:07:09.126432 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39723
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:09.225973 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:09.226276 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:09.226323 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:09.227660 23450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:09.227753 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:09.227782 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:09.227810 23450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:09.227844 23450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:09.230087 23450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:09.230669 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:09.231091 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:09.237339 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:09.237520 23450 server_base.cc:1034] running on GCE node
W20260812 00:07:09.237401 23456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:09.237464 23459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:09.238029 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:09.238466 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:09.239674 23450 hybrid_clock.cc:648] HybridClock initialized: now 1786493229239625 us; error 66 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T00:07:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1623 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:07:09 dist-test-slave-z8x9 krb5kdc[23479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:07:09 dist-test-slave-z8x9 krb5kdc[23479](info): set up 2 sockets
Aug 12 00:07:09 dist-test-slave-z8x9 krb5kdc[23479](info): commencing operation
krb5kdc: starting...
W20260812 00:07:10.641137 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:07:10 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:07:10.789860 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33189
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:10.885263 23508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:10.885516 23508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:10.885542 23508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:10.887122 23508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:10.887193 23508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:10.887210 23508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:10.887223 23508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:10.887236 23508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:10.889276 23508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:10.889600 23508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:10.889880 23508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:10.895933 23516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:10.895905 23517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:10.895905 23520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:10.896766 23508 server_base.cc:1034] running on GCE node
I20260812 00:07:10.897101 23508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:10.897522 23508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:10.898703 23508 hybrid_clock.cc:648] HybridClock initialized: now 1786493230898691 us; error 40 us; skew 500 ppm
Aug 12 00:07:10 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:10.902607 23508 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:07:10.903170 23508 webserver.cc:460] Webserver started at http://127.7.85.254:33789/ using document root <none> and password file <none>
I20260812 00:07:10.903692 23508 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:10.903817 23508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:10.904002 23508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:10.905339 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "61d7c57669a140b7bd1e598dc784627c"
format_stamp: "Formatted at 2026-08-12 00:07:10 on dist-test-slave-z8x9"
server_key: "fcdb3353e07ad9cd1231030ded821f1c"
server_key_iv: "54c0ab83af24a04435c6fc443b5a977e"
server_key_version: "encryptionkey@0"
I20260812 00:07:10.906086 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "61d7c57669a140b7bd1e598dc784627c"
format_stamp: "Formatted at 2026-08-12 00:07:10 on dist-test-slave-z8x9"
server_key: "fcdb3353e07ad9cd1231030ded821f1c"
server_key_iv: "54c0ab83af24a04435c6fc443b5a977e"
server_key_version: "encryptionkey@0"
I20260812 00:07:10.907882 23508 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:10.909015 23539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:10.909421 23508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:10.909561 23508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "61d7c57669a140b7bd1e598dc784627c"
format_stamp: "Formatted at 2026-08-12 00:07:10 on dist-test-slave-z8x9"
server_key: "fcdb3353e07ad9cd1231030ded821f1c"
server_key_iv: "54c0ab83af24a04435c6fc443b5a977e"
server_key_version: "encryptionkey@0"
I20260812 00:07:10.909690 23508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:10.935846 23508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:10.938787 23508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:10.938928 23508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:10.943351 23508 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33189
I20260812 00:07:10.943444 23624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33189 every 8 connection(s)
I20260812 00:07:10.943779 23508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:07:10.944770 23625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:10.947242 23625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Bootstrap starting.
I20260812 00:07:10.947991 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 23508
I20260812 00:07:10.948174 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:07:10.948434 7511 external_mini_cluster.cc:1428] Setting key d6f11979ca50f3e7381b2927c7a83536
I20260812 00:07:10.948984 23625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:10.949633 23625 log.cc:824] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:10.950927 23625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: No bootstrap required, opened a new log
I20260812 00:07:10.953955 23625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:10.954174 23625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:10.954222 23625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d7c57669a140b7bd1e598dc784627c, State: Initialized, Role: FOLLOWER
I20260812 00:07:10.954391 23625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:10.954481 23625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:07:10.954523 23625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:07:10.954591 23625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:10.955345 23625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:10.955503 23625 leader_election.cc:302] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61d7c57669a140b7bd1e598dc784627c; no voters:
I20260812 00:07:10.955744 23625 leader_election.cc:288] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:07:10.955866 23633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:10.956094 23633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 LEADER]: Becoming Leader. State: Replica: 61d7c57669a140b7bd1e598dc784627c, State: Running, Role: LEADER
I20260812 00:07:10.956110 23625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:07:10.956303 23633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:10.956727 23634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61d7c57669a140b7bd1e598dc784627c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } } }
I20260812 00:07:10.956817 23634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:10.956979 23635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61d7c57669a140b7bd1e598dc784627c. Latest consensus state: current_term: 1 leader_uuid: "61d7c57669a140b7bd1e598dc784627c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } } }
I20260812 00:07:10.957041 23635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:10.957543 23648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 00:07:10 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:10.958110 23648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:10.960482 23648 catalog_manager.cc:1347] Generated new cluster ID: 7191571872f64ceca3d3ec183e0533cb
I20260812 00:07:10.960536 23648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:07:10.962770 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:10.949225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.7.85.254:33189)
0812 00:07:10.949518 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:10.949527 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:10.949555 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:07:10.950172 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:10.950180 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:10.950247 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:10.950440 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:10.951864 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:10.953455 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:10.954530 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:10.955035 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:10.955132 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:10.958596 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:10.958613 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:10.958614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:10.958655 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:10.961055 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:10.961484 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:10.961487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:10.961488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:10.961601 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:10.961821 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:10.961822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:10.961822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:10.962082 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:10.962289 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:10.962381 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 00:07:10.962540 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":137,"threads_started":1}
I20260812 00:07:10.978099 23648 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:07:10.978930 23648 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:10.982981 23648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Generated new TSK 0
I20260812 00:07:10.983224 23648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:07:11.029393 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33189
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:11.146569 23671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:11.146817 23671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:11.146849 23671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:11.148118 23671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:11.148164 23671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:11.148198 23671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:07:11.149724 23671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:11.149957 23671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:11.150164 23671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:11.156495 23684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:11.156663 23671 server_base.cc:1034] running on GCE node
W20260812 00:07:11.156617 23688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.156581 23685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:11.157433 23671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:11.157702 23671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:11.158958 23671 hybrid_clock.cc:648] HybridClock initialized: now 1786493231158890 us; error 77 us; skew 500 ppm
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:11.162314 23671 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:07:11.162966 23671 webserver.cc:460] Webserver started at http://127.7.85.193:40209/ using document root <none> and password file <none>
I20260812 00:07:11.163235 23671 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:11.163321 23671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:11.163487 23671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:11.164881 23671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "b278b28699c944d9a4599b4bf9fa1f54"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "f40c6a11abc0461bb9e0e252dfd12ee2"
server_key_iv: "e85f95f764b83871daca93f4036488b8"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.165396 23671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "b278b28699c944d9a4599b4bf9fa1f54"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "f40c6a11abc0461bb9e0e252dfd12ee2"
server_key_iv: "e85f95f764b83871daca93f4036488b8"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.166939 23671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:11.168061 23704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.168354 23671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:11.168485 23671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "b278b28699c944d9a4599b4bf9fa1f54"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "f40c6a11abc0461bb9e0e252dfd12ee2"
server_key_iv: "e85f95f764b83871daca93f4036488b8"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.168573 23671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:11.183813 23671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:11.186715 23671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:11.186926 23671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:11.187207 23671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:11.187567 23671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:11.187626 23671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.187665 23671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:11.187707 23671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.195291 23671 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:33375
I20260812 00:07:11.195400 23904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:33375 every 8 connection(s)
I20260812 00:07:11.195771 23671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:07:11.197567 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 23671
I20260812 00:07:11.197710 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:07:11.197911 7511 external_mini_cluster.cc:1428] Setting key de26403b81ea6c3193cac878f5fb04c8
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:07:11.209069 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:11.196792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:52097 (local address 127.7.85.254:33189)
0812 00:07:11.196887 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:11.196890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:11.197309 (+ 419us) server_negotiation.cc:407] Connection header received
0812 00:07:11.198043 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:11.198044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:11.198059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:11.198131 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:11.199393 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.199932 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.201150 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.201416 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.201500 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:11.204573 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:11.204576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:11.204577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:11.204598 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:11.207174 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:11.207724 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:11.207726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:11.207727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:11.207802 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:11.208146 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:11.208148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:11.208148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:11.208324 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:11.208414 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:11.208799 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 00:07:11.208930 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:07:11.209311 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:11.196756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:52097)
0812 00:07:11.197094 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:11.197191 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:11.197798 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:11.198230 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:11.198234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:11.199050 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:11.199277 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:11.199283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.200059 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.200060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:11.201030 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:11.201033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.201622 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.201631 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:11.201958 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:11.202657 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:11.202668 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:11.204436 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:11.207312 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:11.207323 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:11.207326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:11.207631 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:11.207922 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:11.207923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:11.207924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:11.208061 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:11.208434 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:11.208442 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:11.208624 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 00:07:11.208751 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 00:07:11.208914 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":116,"threads_started":1}
I20260812 00:07:11.210247 23908 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33189
I20260812 00:07:11.210352 23908 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:11.210528 23908 heartbeater.cc:511] Master 127.7.85.254:33189 requested a full tablet report, sending...
I20260812 00:07:11.211166 23564 ts_manager.cc:194] Registered new tserver with Master: b278b28699c944d9a4599b4bf9fa1f54 (127.7.85.193:33375)
I20260812 00:07:11.211848 23564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:52097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:07:11.234534 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33189
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:11.326419 23915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:11.326670 23915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:11.326699 23915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:11.328068 23915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:11.328125 23915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:11.328153 23915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:07:11.329778 23915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:11.330075 23915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:11.330348 23915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:11.336241 23927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.336241 23928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.336308 23930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:11.337280 23915 server_base.cc:1034] running on GCE node
I20260812 00:07:11.337607 23915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:11.337927 23915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:11.339133 23915 hybrid_clock.cc:648] HybridClock initialized: now 1786493231339123 us; error 35 us; skew 500 ppm
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:11.342435 23915 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:07:11.343212 23915 webserver.cc:460] Webserver started at http://127.7.85.194:44791/ using document root <none> and password file <none>
I20260812 00:07:11.343681 23915 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:11.343807 23915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:11.343987 23915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:11.345152 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "a2469d04bf5c44f088d326e2ba18fb55"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "b4dfcaa000eaaa484498525cde8265c8"
server_key_iv: "b6997c21c78f98793f0738bd9e69668e"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.345604 23915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "a2469d04bf5c44f088d326e2ba18fb55"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "b4dfcaa000eaaa484498525cde8265c8"
server_key_iv: "b6997c21c78f98793f0738bd9e69668e"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.347214 23915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:07:11.348050 23939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.348253 23915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:11.348362 23915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "a2469d04bf5c44f088d326e2ba18fb55"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "b4dfcaa000eaaa484498525cde8265c8"
server_key_iv: "b6997c21c78f98793f0738bd9e69668e"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.348453 23915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:11.363747 23915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:11.366607 23915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:11.366806 23915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:11.367102 23915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:11.367460 23915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:11.367517 23915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.367558 23915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:11.367607 23915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.374187 23915 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:45127
I20260812 00:07:11.374233 24153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:45127 every 8 connection(s)
I20260812 00:07:11.374691 23915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:11.382086 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 23915
I20260812 00:07:11.382226 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:07:11.382508 7511 external_mini_cluster.cc:1428] Setting key 9ef5e08a2ac080626eb27876f4a84fe2
I20260812 00:07:11.385774 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:11.375511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:46639)
0812 00:07:11.375702 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:11.375748 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:11.376236 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:11.376583 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:11.376587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:11.377234 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:11.377446 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:11.377451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.378159 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.378160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:11.378978 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:11.378980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.379397 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.379398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:11.379518 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:11.380121 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:11.380133 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:11.381726 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:11.383794 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:11.383801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:11.383804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:11.384141 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:11.384348 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:11.384349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:11.384350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:11.384506 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:11.384854 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:11.384861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:11.385042 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 00:07:11.385204 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 00:07:11.385411 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260812 00:07:11.385825 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:11.375569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:46639 (local address 127.7.85.254:33189)
0812 00:07:11.375755 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:11.375757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:11.375827 (+ 70us) server_negotiation.cc:407] Connection header received
0812 00:07:11.376350 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:11.376352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:11.376364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:11.376438 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:11.377557 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.378063 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.379075 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.379293 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.379360 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:11.381850 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:11.381852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:11.381852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:11.381861 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:11.383708 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:11.384224 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:11.384224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:11.384225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:11.384283 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:11.384611 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:11.384612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:11.384613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:11.384771 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:11.384840 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:11.385418 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 00:07:11.385646 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260812 00:07:11.386351 24154 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33189
I20260812 00:07:11.386468 24154 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:11.386688 24154 heartbeater.cc:511] Master 127.7.85.254:33189 requested a full tablet report, sending...
I20260812 00:07:11.387065 23564 ts_manager.cc:194] Registered new tserver with Master: a2469d04bf5c44f088d326e2ba18fb55 (127.7.85.194:45127)
I20260812 00:07:11.387476 23564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:46639
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:07:11.417591 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33189
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:11.507122 24159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:11.507340 24159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:11.507365 24159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:11.508450 24159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:11.508488 24159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:11.508515 24159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:11.510008 24159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:11.510210 24159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:11.510388 24159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:11.515612 24182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.515784 24174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.515622 24173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:11.515916 24159 server_base.cc:1034] running on GCE node
I20260812 00:07:11.516256 24159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:11.516896 24159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:11.518100 24159 hybrid_clock.cc:648] HybridClock initialized: now 1786493231518127 us; error 91 us; skew 500 ppm
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:11.525743 24159 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:07:11.526237 24159 webserver.cc:460] Webserver started at http://127.7.85.195:32769/ using document root <none> and password file <none>
I20260812 00:07:11.526512 24159 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:11.526587 24159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:11.526739 24159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:11.528043 24159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "923811cdb0ab4d53a05a3134780312a6"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "2b2c58a9b9c6a3151c444d5bd4856a61"
server_key_iv: "6138b986a67974e88bbb36692b22dd75"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.528513 24159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "923811cdb0ab4d53a05a3134780312a6"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "2b2c58a9b9c6a3151c444d5bd4856a61"
server_key_iv: "6138b986a67974e88bbb36692b22dd75"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.530967 24159 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 00:07:11.532061 24192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.532279 24159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:11.532399 24159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "923811cdb0ab4d53a05a3134780312a6"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "2b2c58a9b9c6a3151c444d5bd4856a61"
server_key_iv: "6138b986a67974e88bbb36692b22dd75"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.532485 24159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:11.551730 24159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:11.554796 24159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:11.554980 24159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:11.555269 24159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:11.555663 24159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:11.555729 24159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.555764 24159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:11.555785 24159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.565773 24159 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:33981
I20260812 00:07:11.565817 24401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:33981 every 8 connection(s)
I20260812 00:07:11.566264 24159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:11.575491 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 24159
I20260812 00:07:11.575603 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:07:11.575798 7511 external_mini_cluster.cc:1428] Setting key 0106728393ec893f366e6771feaf404b
I20260812 00:07:11.579164 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:11.567483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:50129 (local address 127.7.85.254:33189)
0812 00:07:11.567548 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:11.567550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:11.567562 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:07:11.568089 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:11.568091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:11.568104 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:11.568183 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:11.569238 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.570186 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.572211 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.572565 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.572603 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:11.575532 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:11.575535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:11.575535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:11.575544 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:11.577531 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:11.578052 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:11.578054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:11.578054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:11.578115 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:11.578425 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:11.578426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:11.578427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:11.578570 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:11.578654 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:11.578986 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 00:07:11.579070 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 00:07:11.579362 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:11.567121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:50129)
0812 00:07:11.567348 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:11.567414 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:11.567991 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:11.568239 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:11.568243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:11.568825 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:11.569077 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:11.569082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.570320 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.570322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:11.572070 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:11.572074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:11.572691 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:11.572692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:11.572806 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:11.573424 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:11.573433 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:11.575377 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:11.577625 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:11.577633 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:11.577636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:11.577958 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:11.578194 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:11.578195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:11.578196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:11.578343 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:11.578644 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:11.578652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:11.578833 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 00:07:11.578950 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 00:07:11.579117 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260812 00:07:11.579859 24402 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33189
I20260812 00:07:11.579975 24402 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:11.580163 24402 heartbeater.cc:511] Master 127.7.85.254:33189 requested a full tablet report, sending...
I20260812 00:07:11.580554 23564 ts_manager.cc:194] Registered new tserver with Master: 923811cdb0ab4d53a05a3134780312a6 (127.7.85.195:33981)
I20260812 00:07:11.580894 23564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:50129
I20260812 00:07:11.587678 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab.
Entry for principal oryx/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab.
WARNING: no policy specified for oryx/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab.
Entry for principal oryx/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab.
WARNING: no policy specified for oryx/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab.
Entry for principal oryx/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab.
I20260812 00:07:11.675797 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 23671
I20260812 00:07:11.690168 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 23915
I20260812 00:07:11.712388 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 24159
I20260812 00:07:11.723230 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 23508
I20260812 00:07:11.733696 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33189
--webserver_interface=127.7.85.254
--webserver_port=33789
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:11.824268 24416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:11.824538 24416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:11.824581 24416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:11.826012 24416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:11.826103 24416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:11.826133 24416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:11.826167 24416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:11.826203 24416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:11.827840 24416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=33789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:11.828142 24416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:11.828385 24416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:11.833552 24423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.833542 24430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.833539 24425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:11.834084 24416 server_base.cc:1034] running on GCE node
I20260812 00:07:11.834321 24416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:11.834576 24416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:11.835716 24416 hybrid_clock.cc:648] HybridClock initialized: now 1786493231835693 us; error 34 us; skew 500 ppm
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:11.839099 24416 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:07:11.839578 24416 webserver.cc:460] Webserver started at http://127.7.85.254:33789/ using document root <none> and password file <none>
I20260812 00:07:11.839838 24416 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:11.839931 24416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:11.841550 24416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:07:11.842329 24443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.842576 24416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:11.842707 24416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "61d7c57669a140b7bd1e598dc784627c"
format_stamp: "Formatted at 2026-08-12 00:07:10 on dist-test-slave-z8x9"
server_key: "fcdb3353e07ad9cd1231030ded821f1c"
server_key_iv: "54c0ab83af24a04435c6fc443b5a977e"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.843072 24416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:11.865152 24416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:11.867880 24416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:11.868060 24416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:11.872339 24416 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33189
I20260812 00:07:11.872380 24528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33189 every 8 connection(s)
I20260812 00:07:11.872778 24416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:07:11.874441 24529 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:07:11.875133 24529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Bootstrap starting.
I20260812 00:07:11.880520 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 24416
I20260812 00:07:11.880915 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:33375
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:33189
--webserver_port=40209
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:07:11.884519 24529 log.cc:824] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:11.886904 24529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:07:11.887269 24529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Bootstrap complete.
I20260812 00:07:11.889771 24529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:11.890012 24529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d7c57669a140b7bd1e598dc784627c, State: Initialized, Role: FOLLOWER
I20260812 00:07:11.890184 24529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:11.890264 24529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:07:11.890329 24529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:07:11.890379 24529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:07:11.891462 24529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:11.891552 24529 leader_election.cc:302] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61d7c57669a140b7bd1e598dc784627c; no voters:
I20260812 00:07:11.891783 24529 leader_election.cc:288] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:07:11.891825 24534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:07:11.891956 24534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [term 2 LEADER]: Becoming Leader. State: Replica: 61d7c57669a140b7bd1e598dc784627c, State: Running, Role: LEADER
I20260812 00:07:11.892046 24534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } }
I20260812 00:07:11.892083 24529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:07:11.892336 24534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "61d7c57669a140b7bd1e598dc784627c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } } }
I20260812 00:07:11.892408 24534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:11.892531 24534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61d7c57669a140b7bd1e598dc784627c. Latest consensus state: current_term: 2 leader_uuid: "61d7c57669a140b7bd1e598dc784627c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d7c57669a140b7bd1e598dc784627c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33189 } } }
I20260812 00:07:11.892606 24534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:11.892731 24544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:07:11.893348 24544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:11.893568 24544 catalog_manager.cc:1259] Loaded cluster ID: 7191571872f64ceca3d3ec183e0533cb
I20260812 00:07:11.893697 24544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:07:11.894897 24544 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:11.895081 24544 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61d7c57669a140b7bd1e598dc784627c: Loaded TSK: 0
I20260812 00:07:11.895308 24544 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:07:11.976436 24532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:11.976713 24532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:11.976754 24532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:11.978083 24532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:11.978575 24532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:11.978696 24532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:07:11.981485 24532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.193.keytab
--principal=oryx/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:33375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=40209
--tserver_master_addrs=127.7.85.254:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:11.981846 24532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:11.982116 24532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:11.988317 24573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.988480 24579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:11.988344 24576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:11.988601 24532 server_base.cc:1034] running on GCE node
I20260812 00:07:11.988803 24532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:11.989105 24532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:11.990309 24532 hybrid_clock.cc:648] HybridClock initialized: now 1786493231990293 us; error 31 us; skew 500 ppm
Aug 12 00:07:11 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493231, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:11.994338 24532 init.cc:377] Logged in from keytab as oryx/127.7.85.193@KRBTEST.COM (short username oryx)
I20260812 00:07:11.994948 24532 webserver.cc:460] Webserver started at http://127.7.85.193:40209/ using document root <none> and password file <none>
I20260812 00:07:11.995276 24532 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:11.995363 24532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:11.997226 24532 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:07:11.998006 24593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:11.998282 24532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:11.998383 24532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "b278b28699c944d9a4599b4bf9fa1f54"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "f40c6a11abc0461bb9e0e252dfd12ee2"
server_key_iv: "e85f95f764b83871daca93f4036488b8"
server_key_version: "encryptionkey@0"
I20260812 00:07:11.998988 24532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:12.040426 24532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:12.043021 24532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:12.043147 24532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:12.043368 24532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:12.043733 24532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:12.043773 24532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.043797 24532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:12.043810 24532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.050343 24532 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:33375
I20260812 00:07:12.050392 24795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:33375 every 8 connection(s)
I20260812 00:07:12.050748 24532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.058804 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 24532
I20260812 00:07:12.059083 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:45127
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:33189
--webserver_port=44791
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:07:12.060231 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.051541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:56433)
0812 00:07:12.051762 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.051807 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.052344 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.052800 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.052804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.053572 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.053779 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.053784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.054936 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.054937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.055769 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.055772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.056393 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.056395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.056510 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.057048 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.057056 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.060006 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":128,"threads_started":1}
W20260812 00:07:12.060328 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.051624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:56433 (local address 127.7.85.254:33189)
0812 00:07:12.051853 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.051863 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.051907 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:07:12.052555 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.052561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.052607 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.052731 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.053900 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.054821 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.055869 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.056292 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.056388 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.060139 (+ 3751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:56433: BlockingRecv error: recv got EOF from 127.7.85.193:56433 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":133,"threads_started":1}
W20260812 00:07:12.060513 24796 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:33189 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.85.254:33189: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.065727 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.060969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:55803)
0812 00:07:12.061041 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.061053 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.061108 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.061280 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.061281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.061416 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.061463 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.061465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.062120 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.062122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.062621 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.062622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.062978 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.062979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.063049 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.063563 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.063568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.065635 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 00:07:12.066062 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.061004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:55803 (local address 127.7.85.254:33189)
0812 00:07:12.061044 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.061046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.061104 (+ 58us) server_negotiation.cc:407] Connection header received
0812 00:07:12.061170 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.061171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.061185 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.061277 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.061563 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.062039 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.062705 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.062903 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.062980 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.065717 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:55803: BlockingRecv error: recv got EOF from 127.7.85.193:55803 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.071311 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.066019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:52947)
0812 00:07:12.066265 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.066277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.066341 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.066506 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.066507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.066684 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.066728 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.066730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.067388 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.067389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.068259 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.068262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.068659 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.068661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.068726 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.069196 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.069202 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.071220 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225}
W20260812 00:07:12.071465 24796 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:07:12.071557 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.066097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:52947 (local address 127.7.85.254:33189)
0812 00:07:12.066178 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.066180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.066338 (+ 158us) server_negotiation.cc:407] Connection header received
0812 00:07:12.066414 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.066415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.066425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.066475 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.066842 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.067298 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.068355 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.068536 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.068640 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.071301 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:52947: BlockingRecv error: recv got EOF from 127.7.85.193:52947 (error 108)
Metrics: {"server-negotiator.queue_time_us":60,"thread_start_us":119,"threads_started":1}
W20260812 00:07:12.153365 24803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:12.153599 24803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:12.153625 24803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:12.154850 24803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:12.154896 24803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:12.154923 24803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:07:12.157400 24803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.194.keytab
--principal=oryx/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:45127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=44791
--tserver_master_addrs=127.7.85.254:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:12.157712 24803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:12.157972 24803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:12.163432 24813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:12.163446 24815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:12.163416 24812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:12.163682 24803 server_base.cc:1034] running on GCE node
I20260812 00:07:12.163924 24803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:12.164237 24803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:12.165449 24803 hybrid_clock.cc:648] HybridClock initialized: now 1786493232165425 us; error 45 us; skew 500 ppm
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:12.168360 24803 init.cc:377] Logged in from keytab as oryx/127.7.85.194@KRBTEST.COM (short username oryx)
I20260812 00:07:12.168910 24803 webserver.cc:460] Webserver started at http://127.7.85.194:44791/ using document root <none> and password file <none>
I20260812 00:07:12.169220 24803 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:12.169324 24803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:12.170931 24803 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:12.171744 24823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.172256 24803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:12.172402 24803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "a2469d04bf5c44f088d326e2ba18fb55"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "b4dfcaa000eaaa484498525cde8265c8"
server_key_iv: "b6997c21c78f98793f0738bd9e69668e"
server_key_version: "encryptionkey@0"
I20260812 00:07:12.172838 24803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:12.196714 24803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:12.199707 24803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:12.199905 24803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:12.200201 24803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:12.200594 24803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:12.200655 24803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.200836 24803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:12.200934 24803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.208935 24803 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:45127
I20260812 00:07:12.209017 25033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:45127 every 8 connection(s)
I20260812 00:07:12.209460 24803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:07:12.216478 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 24803
I20260812 00:07:12.216851 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:33981
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:33189
--webserver_port=32769
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 00:07:12.221716 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.210416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:46275 (local address 127.7.85.254:33189)
0812 00:07:12.210507 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.210509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.210601 (+ 92us) server_negotiation.cc:407] Connection header received
0812 00:07:12.211114 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.211116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.211139 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.211241 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.212225 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.213046 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.214720 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.215088 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.215175 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.221625 (+ 6450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:46275: BlockingRecv error: recv got EOF from 127.7.85.194:46275 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:07:12.222283 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.210303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:46275)
0812 00:07:12.210482 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.210530 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.211017 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.211302 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.211306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.211915 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.212119 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.212124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.213200 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.213202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.214608 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.214611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.215187 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.215188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.215429 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.216193 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.216205 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.222018 (+ 5813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
W20260812 00:07:12.222949 25034 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:33189 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.85.254:33189: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.232052 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.225469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:39219)
0812 00:07:12.225527 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.225539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.225592 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.225784 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.225785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.225936 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.225996 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.225997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.226864 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.226865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.227853 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.227857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.228487 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.228488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.228576 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.229125 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.229130 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.231990 (+ 2860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 00:07:12.232245 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.225543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:39219 (local address 127.7.85.254:33189)
0812 00:07:12.225608 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.225611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.225623 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:07:12.225673 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.225675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.225687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.225770 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.226095 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.226736 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.228010 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.228357 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.228450 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.232147 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:39219: BlockingRecv error: recv got EOF from 127.7.85.194:39219 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.238396 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.232623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:59665)
0812 00:07:12.232760 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.232771 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.232831 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.233009 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.233010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.233135 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.233218 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.233220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.234328 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.234329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.234833 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.234834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.235360 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.235361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.235402 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.235980 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.235989 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.238336 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260812 00:07:12.238554 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.232690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:59665 (local address 127.7.85.254:33189)
0812 00:07:12.232733 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.232735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.232832 (+ 97us) server_negotiation.cc:407] Connection header received
0812 00:07:12.232887 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.232888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.232899 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.232974 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.233332 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.234190 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.234921 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.235258 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.235334 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.238485 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:59665: BlockingRecv error: recv got EOF from 127.7.85.194:59665 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 00:07:12.238574 25034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:07:12.320904 25037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:12.321161 25037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:12.321210 25037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:12.322610 25037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:12.322679 25037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:12.322717 25037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:12.324381 25037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/krb5kdc/oryx_127.7.85.195.keytab
--principal=oryx/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:33981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=32769
--tserver_master_addrs=127.7.85.254:33189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:12.324668 25037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:12.324966 25037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:12.330295 25049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:12.330416 25037 server_base.cc:1034] running on GCE node
W20260812 00:07:12.330242 25055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:12.330235 25046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:12.330703 25037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:12.330919 25037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:12.332108 25037 hybrid_clock.cc:648] HybridClock initialized: now 1786493232332061 us; error 78 us; skew 500 ppm
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:12.335031 25037 init.cc:377] Logged in from keytab as oryx/127.7.85.195@KRBTEST.COM (short username oryx)
I20260812 00:07:12.335526 25037 webserver.cc:460] Webserver started at http://127.7.85.195:32769/ using document root <none> and password file <none>
I20260812 00:07:12.335811 25037 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:12.335911 25037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:12.337455 25037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:12.338176 25071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.338409 25037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:12.338522 25037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "923811cdb0ab4d53a05a3134780312a6"
format_stamp: "Formatted at 2026-08-12 00:07:11 on dist-test-slave-z8x9"
server_key: "2b2c58a9b9c6a3151c444d5bd4856a61"
server_key_iv: "6138b986a67974e88bbb36692b22dd75"
server_key_version: "encryptionkey@0"
I20260812 00:07:12.338893 25037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:12.356285 25037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:12.358796 25037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:12.359001 25037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:12.359283 25037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:12.359635 25037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:12.359740 25037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.359787 25037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:12.359814 25037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:12.366410 25037 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:33981
I20260812 00:07:12.366449 25285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:33981 every 8 connection(s)
I20260812 00:07:12.366871 25037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.375113 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.367877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:42789)
0812 00:07:12.368063 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.368108 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.368652 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.368978 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.368981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.369648 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.369864 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.369869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.370557 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.370557 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.371403 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.371405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.371837 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.371839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.371966 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.372488 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.372496 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.374947 (+ 2451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
W20260812 00:07:12.375114 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.367856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:42789 (local address 127.7.85.254:33189)
0812 00:07:12.367963 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.367965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.368230 (+ 265us) server_negotiation.cc:407] Connection header received
0812 00:07:12.368795 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.368796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.368808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.368868 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.369964 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.370470 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.371529 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.371725 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.371791 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.375063 (+ 3272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:42789: BlockingRecv error: recv got EOF from 127.7.85.195:42789 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 00:07:12.375388 25286 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:33189 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.85.254:33189: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
I20260812 00:07:12.375936 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 25037
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.381466 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.375641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:58063)
0812 00:07:12.375696 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.375707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.375747 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.376121 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.376122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.376345 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.376407 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.376410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.377484 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.377485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.378266 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.378268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.378756 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.378757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.378807 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.379289 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.379300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.381387 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 00:07:12.382006 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.375945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:58063 (local address 127.7.85.254:33189)
0812 00:07:12.375985 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.375987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.375996 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:12.376021 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.376021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.376030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.376099 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.376492 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.377395 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.378360 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.378669 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.378749 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.381922 (+ 3173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:58063: BlockingRecv error: recv got EOF from 127.7.85.195:58063 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:07:12.385408 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.376396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.7.85.254:33189)
0812 00:07:12.376456 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.376458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.376632 (+ 174us) server_negotiation.cc:407] Connection header received
0812 00:07:12.376727 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.376744 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.376754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.376812 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.377651 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.378396 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.379272 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.379576 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.379689 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.381136 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:12.381141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:12.381142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:12.381155 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:12.383790 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:12.384289 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:12.384295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:12.384320 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:12.384421 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:12.384655 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:12.384657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:12.384657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:12.384898 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:12.385047 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:12.385164 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 00:07:12.385329 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:12 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:12.388293 25290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:12.382642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:43259)
0812 00:07:12.383904 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:12.383922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:12.383986 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:12.384115 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:12.384116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:12.384262 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:12.384292 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.384293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.384913 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.384914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.385638 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:12.385641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.386020 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.386021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:12.386077 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.386508 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:12.386513 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:12.388239 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1245}
W20260812 00:07:12.388446 24799 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:12.382705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:43259 (local address 127.7.85.254:33189)
0812 00:07:12.382765 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:12.382767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:12.383972 (+ 1205us) server_negotiation.cc:407] Connection header received
0812 00:07:12.384044 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:12.384045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:12.384055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:12.384110 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:12.384368 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.384830 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.385714 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:12.385943 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:12.386008 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:12.388364 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:43259: BlockingRecv error: recv got EOF from 127.7.85.195:43259 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 00:07:12.388439 25286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 00:07:13 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:13 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:13.077869 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:13.071947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:48753)
0812 00:07:13.072134 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:13.072155 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:13.072219 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:13.072419 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:13.072420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:13.072598 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:13.072649 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:13.072651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.073452 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.073453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:13.074006 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:13.074008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.074491 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.074493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:13.074545 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:13.075158 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:13.075164 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:13.077791 (+ 2627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
W20260812 00:07:13.078096 25295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:13.072156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:48753 (local address 127.7.85.254:33189)
0812 00:07:13.072286 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:13.072288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:13.072297 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:13.072324 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:13.072325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:13.072335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:13.072392 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:13.072744 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.073344 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.074159 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.074382 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.074449 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:13.078007 (+ 3558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:48753: BlockingRecv error: recv got EOF from 127.7.85.193:48753 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
Aug 12 00:07:13 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:13 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:13.245826 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:13.239175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:32819)
0812 00:07:13.239719 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:13.239735 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:13.239856 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:13.240068 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:13.240069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:13.240327 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:13.240396 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:13.240398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.241480 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.241482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:13.242134 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:13.242137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.242595 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.242596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:13.242777 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:13.243368 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:13.243374 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:13.245733 (+ 2359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":149,"threads_started":1}
W20260812 00:07:13.245903 25295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:13.239316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:32819 (local address 127.7.85.254:33189)
0812 00:07:13.239705 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:13.239709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:13.239828 (+ 119us) server_negotiation.cc:407] Connection header received
0812 00:07:13.239942 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:13.239943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:13.239964 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:13.240058 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:13.240493 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.241338 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.242260 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.242509 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.242597 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:13.245848 (+ 3251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:32819: BlockingRecv error: recv got EOF from 127.7.85.194:32819 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Aug 12 00:07:13 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:13 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:13.394228 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:13.388907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:58005)
0812 00:07:13.389122 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:13.389138 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:13.389298 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:13.389475 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:13.389476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:13.389680 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:13.389729 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:13.389731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.390455 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.390456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:13.391015 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:13.391016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.391395 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.391396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:13.391456 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:13.392086 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:13.392092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:13.394130 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
W20260812 00:07:13.394397 25295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:13.389031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:58005 (local address 127.7.85.254:33189)
0812 00:07:13.389128 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:13.389130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:13.389252 (+ 122us) server_negotiation.cc:407] Connection header received
0812 00:07:13.389375 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:13.389376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:13.389398 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:13.389468 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:13.389832 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.390355 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.391102 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:13.391300 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:13.391381 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:13.394310 (+ 2929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:58005: BlockingRecv error: recv got EOF from 127.7.85.195:58005 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 00:07:14 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:14 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:14.083592 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:14.078409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:40739)
0812 00:07:14.078593 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:14.078607 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:14.078666 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:14.078822 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:14.078823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:14.079018 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:14.079071 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:14.079072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.079768 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.079769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:14.080363 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:14.080365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.080749 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.080751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:14.080811 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:14.081404 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:14.081410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:14.083530 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
W20260812 00:07:14.083806 25299 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:14.078521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:40739 (local address 127.7.85.254:33189)
0812 00:07:14.078661 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:14.078663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:14.078673 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:14.078744 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:14.078744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:14.078755 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:14.078809 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:14.079161 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.079687 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.080445 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.080644 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.080702 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:14.083713 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:40739: BlockingRecv error: recv got EOF from 127.7.85.193:40739 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
Aug 12 00:07:14 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:14 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:14.251824 25301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:14.246491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:43805)
0812 00:07:14.246780 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:14.246792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:14.246895 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:14.247134 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:14.247136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:14.247365 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:14.247416 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:14.247418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.248105 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.248106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:14.248647 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:14.248649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.249117 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.249118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:14.249233 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:14.249775 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:14.249780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:14.251703 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":93,"threads_started":1}
W20260812 00:07:14.251855 25299 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:14.246554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:43805 (local address 127.7.85.254:33189)
0812 00:07:14.246755 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:14.246757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:14.246873 (+ 116us) server_negotiation.cc:407] Connection header received
0812 00:07:14.246990 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:14.246991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:14.247003 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:14.247053 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:14.247507 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.248015 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.248790 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.249009 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.249082 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:14.251789 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:43805: BlockingRecv error: recv got EOF from 127.7.85.194:43805 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
Aug 12 00:07:14 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:14 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:14.400717 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:14.394874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:60581)
0812 00:07:14.395012 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:14.395026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:14.395100 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:14.395282 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:14.395283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:14.395536 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:14.395611 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:14.395614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.396384 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.396385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:14.397025 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:14.397028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.397791 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.397793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:14.397842 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:14.398533 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:14.398539 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:14.400654 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
W20260812 00:07:14.400906 25299 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:14.394982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:60581 (local address 127.7.85.254:33189)
0812 00:07:14.395035 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:14.395037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:14.395103 (+ 66us) server_negotiation.cc:407] Connection header received
0812 00:07:14.395169 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:14.395170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:14.395186 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:14.395247 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:14.395728 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.396288 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.397415 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:14.397629 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:14.397706 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:14.400826 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:60581: BlockingRecv error: recv got EOF from 127.7.85.195:60581 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 00:07:15 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:15 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:15.089530 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:15.084204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:35233)
0812 00:07:15.084364 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:15.084382 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:15.084437 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:15.084730 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:15.084732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:15.084939 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:15.085005 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:15.085007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.085801 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.085802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:15.086440 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:15.086443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.086776 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.086777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:15.086821 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:15.087360 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:15.087365 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:15.089455 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
W20260812 00:07:15.089731 25305 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:15.084393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:35233 (local address 127.7.85.254:33189)
0812 00:07:15.084530 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:15.084532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:15.084550 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:07:15.084578 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:15.084579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:15.084591 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:15.084663 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:15.085096 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.085684 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.086516 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.086710 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.086768 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:15.089618 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:35233: BlockingRecv error: recv got EOF from 127.7.85.193:35233 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
Aug 12 00:07:15 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:15 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:15.257678 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:15.252493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:36071)
0812 00:07:15.252642 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:15.252659 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:15.252733 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:15.252914 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:15.252915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:15.253113 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:15.253162 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:15.253164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.253979 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.253979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:15.254561 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:15.254563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.254963 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.254964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:15.255032 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:15.255519 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:15.255524 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:15.257569 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260812 00:07:15.257730 25305 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:15.252614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:36071 (local address 127.7.85.254:33189)
0812 00:07:15.252725 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:15.252727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:15.252741 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:07:15.252814 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:15.252815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:15.252835 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:15.252907 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:15.253310 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.253861 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.254650 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.254873 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.254963 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:15.257661 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:36071: BlockingRecv error: recv got EOF from 127.7.85.194:36071 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 00:07:15 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:15 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:15.408258 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:15.401477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:38845)
0812 00:07:15.401652 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:15.401668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:15.401774 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:15.401993 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:15.401994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:15.402205 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:15.402272 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:15.402275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.403150 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.403152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:15.404527 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:15.404531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.404999 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.405000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:15.405058 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:15.405719 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:15.405724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:15.408141 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
W20260812 00:07:15.408362 25305 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:15.401630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:38845 (local address 127.7.85.254:33189)
0812 00:07:15.401702 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:15.401705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:15.401721 (+ 16us) server_negotiation.cc:407] Connection header received
0812 00:07:15.401887 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:15.401888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:15.401904 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:15.401968 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:15.402381 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.403026 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.404649 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:15.404918 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:15.404999 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:15.408293 (+ 3294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:38845: BlockingRecv error: recv got EOF from 127.7.85.195:38845 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 00:07:16 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:16 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:16.095593 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:16.090115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:38641)
0812 00:07:16.090287 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:16.090298 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:16.090352 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:16.090573 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:16.090575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:16.090770 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:16.090822 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:16.090824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.091704 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.091705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:16.092414 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:16.092418 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.092944 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.092945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:16.092995 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:16.093544 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:16.093549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:16.095528 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
W20260812 00:07:16.095783 25315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:16.090227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:38641 (local address 127.7.85.254:33189)
0812 00:07:16.090410 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:16.090411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:16.090421 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:16.090448 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:16.090449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:16.090460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:16.090529 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:16.090903 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.091585 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.092506 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.092823 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.092934 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:16.095684 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:38641: BlockingRecv error: recv got EOF from 127.7.85.193:38641 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":115,"threads_started":1}
Aug 12 00:07:16 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:16 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:16.263621 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:16.258286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:60383)
0812 00:07:16.258439 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:16.258454 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:16.258536 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:16.258740 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:16.258741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:16.259105 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:16.259170 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:16.259173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.259933 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.259934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:16.260528 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:16.260530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.261065 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.261066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:16.261129 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:16.261726 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:16.261730 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:16.263559 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260812 00:07:16.263823 25315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:16.258410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:60383 (local address 127.7.85.254:33189)
0812 00:07:16.258464 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:16.258466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:16.258529 (+ 63us) server_negotiation.cc:407] Connection header received
0812 00:07:16.258621 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:16.258623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:16.258643 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:16.258718 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:16.259311 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.259839 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.260640 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.260980 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.261055 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:16.263743 (+ 2688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:60383: BlockingRecv error: recv got EOF from 127.7.85.194:60383 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 00:07:16 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:16 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:16.413982 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:16.408873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:47465)
0812 00:07:16.409074 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:16.409087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:16.409157 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:16.409334 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:16.409335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:16.409544 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:16.409597 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:16.409598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.410340 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.410341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:16.410930 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:16.410931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.411342 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.411360 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:16.411403 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:16.411936 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:16.411941 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:16.413919 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
W20260812 00:07:16.414099 25315 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:16.408962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:47465 (local address 127.7.85.254:33189)
0812 00:07:16.409069 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:16.409071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:16.409147 (+ 76us) server_negotiation.cc:407] Connection header received
0812 00:07:16.409240 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:16.409241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:16.409256 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:16.409324 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:16.409702 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.410253 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.411028 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:16.411259 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:16.411328 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:16.414021 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:47465: BlockingRecv error: recv got EOF from 127.7.85.195:47465 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 00:07:17 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:17 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:17.105090 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:17.096094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:50729)
0812 00:07:17.096307 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:17.096319 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:17.096377 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:17.096549 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:17.096551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:17.096739 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:17.096791 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:17.096792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.097595 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.097596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:17.098137 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:17.098139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.098544 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.098545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:17.098597 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:17.102697 (+ 4100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:17.102702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:17.105027 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":131,"threads_started":1}
W20260812 00:07:17.105324 25324 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:17.096195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:50729 (local address 127.7.85.254:33189)
0812 00:07:17.096380 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:17.096382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:17.096410 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:07:17.096434 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:17.096435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:17.096446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:17.096495 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:17.096893 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.097517 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.098263 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.098463 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.098541 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:17.105237 (+ 6696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:50729: BlockingRecv error: recv got EOF from 127.7.85.193:50729 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":122,"threads_started":1}
Aug 12 00:07:17 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:17 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:17.269536 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:17.264235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:58891)
0812 00:07:17.264395 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:17.264413 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:17.264489 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:17.264685 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:17.264686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:17.264876 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:17.264922 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:17.264924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.265670 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.265671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:17.266203 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:17.266204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.266568 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.266569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:17.266634 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:17.267199 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:17.267205 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:17.269457 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260812 00:07:17.269660 25324 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:17.264434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:58891 (local address 127.7.85.254:33189)
0812 00:07:17.264510 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:17.264512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:17.264522 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:17.264556 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:17.264559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:17.264574 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:17.264651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:17.265006 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.265581 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.266292 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.266491 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.266571 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:17.269561 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:58891: BlockingRecv error: recv got EOF from 127.7.85.194:58891 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 00:07:17 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:17 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:17.420048 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:17.414628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:55699)
0812 00:07:17.414775 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:17.414794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:17.414848 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:17.414994 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:17.414995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:17.415172 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:17.415230 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:17.415231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.416265 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.416266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:17.416800 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:17.416802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.417400 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.417401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:17.417450 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:17.417945 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:17.417950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:17.419985 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":108,"threads_started":1}
W20260812 00:07:17.420266 25324 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:17.414771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:55699 (local address 127.7.85.254:33189)
0812 00:07:17.414844 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:17.414845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:17.414859 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:07:17.414892 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:17.414893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:17.414907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:17.414974 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:17.415471 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.416193 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.417078 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:17.417316 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:17.417383 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:17.420120 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:55699: BlockingRecv error: recv got EOF from 127.7.85.195:55699 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 00:07:18 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:18 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:18.111433 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:18.105613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:40731)
0812 00:07:18.105778 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:18.105790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:18.105844 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:18.106142 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:18.106143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:18.106332 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:18.106384 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:18.106385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.107130 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.107132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:18.107747 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:18.107750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.108421 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.108423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:18.108474 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:18.109144 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:18.109149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:18.111346 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
W20260812 00:07:18.111555 25330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:18.105837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:40731 (local address 127.7.85.254:33189)
0812 00:07:18.105974 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:18.105976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:18.105985 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:18.106017 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:18.106018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:18.106030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:18.106097 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:18.106495 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.107033 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.107999 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.108283 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.108399 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:18.111476 (+ 3077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:40731: BlockingRecv error: recv got EOF from 127.7.85.193:40731 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
Aug 12 00:07:18 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:18 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:18.275830 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:18.270126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:37507)
0812 00:07:18.270335 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:18.270348 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:18.270421 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:18.270594 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:18.270596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:18.270786 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:18.270837 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:18.270839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.271933 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.271935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:18.272506 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:18.272512 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.272928 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.272930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:18.272988 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:18.273576 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:18.273582 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:18.275765 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
W20260812 00:07:18.276106 25330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:18.270226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:37507 (local address 127.7.85.254:33189)
0812 00:07:18.270337 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:18.270339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:18.270416 (+ 77us) server_negotiation.cc:407] Connection header received
0812 00:07:18.270485 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:18.270486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:18.270497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:18.270543 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:18.270959 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.271820 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.272609 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.272825 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.272887 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:18.275938 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:37507: BlockingRecv error: recv got EOF from 127.7.85.194:37507 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 12 00:07:18 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:18 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:18.427560 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:18.421231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:40565)
0812 00:07:18.421555 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:18.421652 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:18.421774 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:18.421968 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:18.421970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:18.422184 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:18.422233 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:18.422235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.423110 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.423112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:18.423650 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:18.423652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.424032 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.424033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:18.424101 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:18.425074 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:18.425079 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:18.427450 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":105,"threads_started":1}
W20260812 00:07:18.427620 25330 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:18.421269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:40565 (local address 127.7.85.254:33189)
0812 00:07:18.421381 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:18.421383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:18.421761 (+ 378us) server_negotiation.cc:407] Connection header received
0812 00:07:18.421857 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:18.421858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:18.421874 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:18.421944 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:18.422321 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.422984 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.423762 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:18.423955 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:18.424032 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:18.427550 (+ 3518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:40565: BlockingRecv error: recv got EOF from 127.7.85.195:40565 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 00:07:19 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:19 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:19.117359 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:19.112047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:43111)
0812 00:07:19.112228 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:19.112240 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:19.112310 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:19.112493 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:19.112494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:19.112665 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:19.112724 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:19.112726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.113507 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.113508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:19.114111 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:19.114113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.114509 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.114510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:19.114560 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:19.115091 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:19.115096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:19.117295 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":127,"threads_started":1}
W20260812 00:07:19.117511 25334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:19.112122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:43111 (local address 127.7.85.254:33189)
0812 00:07:19.112272 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:19.112274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:19.112288 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:07:19.112359 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:19.112360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:19.112371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:19.112439 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:19.112811 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.113411 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.114215 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.114426 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.114486 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:19.117412 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:43111: BlockingRecv error: recv got EOF from 127.7.85.193:43111 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":126,"threads_started":1}
Aug 12 00:07:19 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:19 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:19.281599 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:19.276405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:36595)
0812 00:07:19.276602 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:19.276614 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:19.276715 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:19.276902 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:19.276903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:19.277159 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:19.277233 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:19.277235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.277945 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.277946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:19.278477 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:19.278479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.278868 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.278869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:19.278930 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:19.279435 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:19.279439 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:19.281523 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
W20260812 00:07:19.281692 25334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:19.276473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:36595 (local address 127.7.85.254:33189)
0812 00:07:19.276561 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:19.276564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:19.276695 (+ 131us) server_negotiation.cc:407] Connection header received
0812 00:07:19.276799 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:19.276800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:19.276820 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:19.276880 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:19.277338 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.277849 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.278565 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.278773 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.278855 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:19.281640 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:36595: BlockingRecv error: recv got EOF from 127.7.85.194:36595 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 00:07:19 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:19 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:19.433476 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:19.428201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:49327)
0812 00:07:19.428386 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:19.428398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:19.428458 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:19.428653 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:19.428655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:19.428860 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:19.428914 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:19.428916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.429623 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.429624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:19.430207 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:19.430209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.430624 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.430625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:19.430674 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:19.431211 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:19.431216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:19.433413 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":71,"threads_started":1}
W20260812 00:07:19.433593 25334 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:19.428335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:49327 (local address 127.7.85.254:33189)
0812 00:07:19.428411 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:19.428414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:19.428444 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:07:19.428519 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:19.428520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:19.428533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:19.428595 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:19.429022 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.429547 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.430335 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:19.430528 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:19.430590 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:19.433513 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:49327: BlockingRecv error: recv got EOF from 127.7.85.195:49327 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 00:07:20 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:20 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:20.124049 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:20.117998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:34469)
0812 00:07:20.118277 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:20.118305 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:20.118406 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:20.118583 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:20.118585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:20.118762 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:20.118814 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:20.118816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.119560 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.119561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:20.120102 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:20.120104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.120531 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.120532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:20.120590 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:20.121245 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:20.121252 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:20.123981 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":83,"threads_started":1}
W20260812 00:07:20.124203 25342 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:20.118240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:34469 (local address 127.7.85.254:33189)
0812 00:07:20.118388 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:20.118390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:20.118399 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:20.118469 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:20.118470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:20.118481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:20.118534 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:20.118923 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.119478 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.120202 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.120410 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.120824 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:20.124102 (+ 3278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:34469: BlockingRecv error: recv got EOF from 127.7.85.193:34469 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
Aug 12 00:07:20 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:20 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:20.288429 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:20.282560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:53213)
0812 00:07:20.282865 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:20.282876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:20.282950 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:20.283149 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:20.283150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:20.283358 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:20.283429 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:20.283431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.284669 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.284670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:20.285241 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:20.285242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.285687 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.285688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:20.285741 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:20.286379 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:20.286385 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:20.288323 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":110,"threads_started":1}
W20260812 00:07:20.288477 25342 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:20.282669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53213 (local address 127.7.85.254:33189)
0812 00:07:20.282751 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:20.282754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:20.282978 (+ 224us) server_negotiation.cc:407] Connection header received
0812 00:07:20.283030 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:20.283031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:20.283044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:20.283129 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:20.283524 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.284526 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.285357 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.285564 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.285632 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:20.288410 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:53213: BlockingRecv error: recv got EOF from 127.7.85.194:53213 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 00:07:20 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:20 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:20.439483 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:20.434062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:36645)
0812 00:07:20.434269 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:20.434281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:20.434354 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:20.434500 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:20.434500 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:20.434681 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:20.434734 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:20.434735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.435506 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.435507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:20.436053 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:20.436055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.436482 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.436483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:20.436527 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:20.437263 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:20.437268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:20.439398 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
W20260812 00:07:20.439617 25342 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:20.434217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:36645 (local address 127.7.85.254:33189)
0812 00:07:20.434273 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:20.434276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:20.434379 (+ 103us) server_negotiation.cc:407] Connection header received
0812 00:07:20.434413 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:20.434414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:20.434429 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:20.434493 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:20.434835 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.435387 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.436163 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:20.436381 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:20.436455 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:20.439506 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:36645: BlockingRecv error: recv got EOF from 127.7.85.195:36645 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 00:07:21 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:21 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:21.129884 25350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:21.124711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:48357)
0812 00:07:21.124868 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:21.124888 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:21.124938 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:21.125115 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:21.125116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:21.125318 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:21.125395 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:21.125396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.126099 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.126100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:21.126651 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:21.126653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.127062 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.127063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:21.127109 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:21.127608 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:21.127613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:21.129804 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
W20260812 00:07:21.130010 25354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:21.124827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:48357 (local address 127.7.85.254:33189)
0812 00:07:21.124969 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:21.124971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:21.124980 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:21.125012 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:21.125012 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:21.125022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:21.125089 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:21.125474 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.125978 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.126772 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.126976 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.127041 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:21.129921 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:48357: BlockingRecv error: recv got EOF from 127.7.85.193:48357 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
Aug 12 00:07:21 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:21 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:21.295305 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:21.289149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:47143)
0812 00:07:21.289467 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:21.289483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:21.289716 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:21.290051 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:21.290053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:21.290301 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:21.290367 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:21.290369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.291091 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.291093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:21.291639 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:21.291641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.292068 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.292069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:21.292132 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:21.292809 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:21.292817 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:21.295218 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":109,"threads_started":1}
W20260812 00:07:21.295552 25354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:21.289386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:47143 (local address 127.7.85.254:33189)
0812 00:07:21.289548 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:21.289558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:21.289622 (+ 64us) server_negotiation.cc:407] Connection header received
0812 00:07:21.289881 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:21.289883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:21.289904 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:21.289995 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:21.290462 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.290984 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.291748 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.291962 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.292045 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:21.295360 (+ 3315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:47143: BlockingRecv error: recv got EOF from 127.7.85.194:47143 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 00:07:21 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:21 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:21.445369 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:21.440025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:41633)
0812 00:07:21.440183 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:21.440220 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:21.440290 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:21.440424 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:21.440425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:21.440594 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:21.440645 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:21.440647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.441401 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.441402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:21.441943 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:21.441945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.442372 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.442374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:21.442429 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:21.443041 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:21.443046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:21.445301 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260812 00:07:21.445498 25354 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:21.440104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:41633 (local address 127.7.85.254:33189)
0812 00:07:21.440212 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:21.440214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:21.440283 (+ 69us) server_negotiation.cc:407] Connection header received
0812 00:07:21.440335 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:21.440336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:21.440348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:21.440406 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:21.440726 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.441325 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.442023 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:21.442259 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:21.442319 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:21.445394 (+ 3075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:41633: BlockingRecv error: recv got EOF from 127.7.85.195:41633 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 00:07:22 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:22 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:22.141530 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:22.130429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:36293)
0812 00:07:22.130580 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:22.130593 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:22.130650 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:22.130821 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:22.130822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:22.130995 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:22.131048 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:22.131050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.131721 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.131723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:22.132265 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:22.132267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.132656 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.132656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:22.132699 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:22.139525 (+ 6826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:22.139531 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:22.141466 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260812 00:07:22.141706 25358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:22.130519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:36293 (local address 127.7.85.254:33189)
0812 00:07:22.130643 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:22.130645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:22.130659 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:07:22.130716 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:22.130716 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:22.130727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:22.130794 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:22.131115 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.131628 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.132363 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.132576 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.132647 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:22.141591 (+ 8944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:36293: BlockingRecv error: recv got EOF from 127.7.85.193:36293 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
Aug 12 00:07:22 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:22 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:22.301371 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:22.295993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:45513)
0812 00:07:22.296215 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:22.296227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:22.296337 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:22.296513 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:22.296514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:22.296799 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:22.296859 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:22.296861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.297724 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.297725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:22.298252 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:22.298253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.298663 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.298664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:22.298721 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:22.299237 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:22.299242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:22.301287 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
W20260812 00:07:22.301445 25358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:22.296113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:45513 (local address 127.7.85.254:33189)
0812 00:07:22.296192 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:22.296194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:22.296314 (+ 120us) server_negotiation.cc:407] Connection header received
0812 00:07:22.296423 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:22.296424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:22.296443 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:22.296505 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:22.297025 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.297640 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.298372 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.298566 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.298663 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:22.301393 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:45513: BlockingRecv error: recv got EOF from 127.7.85.194:45513 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 00:07:22 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:22 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:22.450843 25362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:22.445930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:40921)
0812 00:07:22.446087 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:22.446101 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:22.446178 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:22.446358 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:22.446360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:22.446553 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:22.446603 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:22.446605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.447360 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.447361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:22.447893 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:22.447895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.448260 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.448261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:22.448327 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:22.448843 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:22.448848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:22.450720 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
W20260812 00:07:22.450870 25358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:22.446139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:40921 (local address 127.7.85.254:33189)
0812 00:07:22.446211 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:22.446214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:22.446223 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:22.446268 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:22.446269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:22.446282 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:22.446347 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:22.446691 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.447261 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.447984 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:22.448185 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:22.448258 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:22.450802 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:40921: BlockingRecv error: recv got EOF from 127.7.85.195:40921 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 00:07:23 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:23 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:23.147915 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:23.142280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:52633)
0812 00:07:23.142755 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:23.142773 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:23.142916 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:23.143114 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:23.143115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:23.143339 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:23.143413 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:23.143416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.144163 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.144165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:23.144724 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:23.144726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.145079 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.145080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:23.145140 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:23.145823 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:23.145830 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:23.147796 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":246,"threads_started":1}
W20260812 00:07:23.147962 25366 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:23.142522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:52633 (local address 127.7.85.254:33189)
0812 00:07:23.142652 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:23.142654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:23.142869 (+ 215us) server_negotiation.cc:407] Connection header received
0812 00:07:23.143029 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:23.143030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:23.143048 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:23.143109 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:23.143509 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.144073 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.144805 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.145005 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.145080 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:23.147884 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:52633: BlockingRecv error: recv got EOF from 127.7.85.193:52633 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
Aug 12 00:07:23 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:23 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:23.307283 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:23.302124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:52627)
0812 00:07:23.302266 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:23.302278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:23.302351 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:23.302494 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:23.302494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:23.302695 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:23.302744 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:23.302746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.303458 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.303459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:23.304102 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:23.304106 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.304563 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.304564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:23.304631 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:23.305136 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:23.305141 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:23.307217 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260812 00:07:23.307472 25366 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:23.302145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:52627 (local address 127.7.85.254:33189)
0812 00:07:23.302195 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:23.302198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:23.302362 (+ 164us) server_negotiation.cc:407] Connection header received
0812 00:07:23.302397 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:23.302398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:23.302412 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:23.302489 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:23.302840 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.303350 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.304206 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.304416 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.304560 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:23.307395 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:52627: BlockingRecv error: recv got EOF from 127.7.85.194:52627 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 00:07:23 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:23 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:23.457867 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:23.451609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:54729)
0812 00:07:23.451930 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:23.451953 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:23.452073 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:23.452247 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:23.452248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:23.452458 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:23.452508 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:23.452510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.453224 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.453225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:23.453773 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:23.453775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.454298 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.454299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:23.454372 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:23.454948 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:23.454953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:23.457736 (+ 2783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":164,"threads_started":1}
W20260812 00:07:23.457913 25366 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:23.451773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:54729 (local address 127.7.85.254:33189)
0812 00:07:23.451902 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:23.451905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:23.452042 (+ 137us) server_negotiation.cc:407] Connection header received
0812 00:07:23.452147 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:23.452148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:23.452169 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:23.452240 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:23.452586 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.453093 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.453875 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:23.454172 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:23.454295 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:23.457841 (+ 3546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:54729: BlockingRecv error: recv got EOF from 127.7.85.195:54729 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 00:07:24 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:24 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:24.153764 25369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:24.148501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:59841)
0812 00:07:24.148698 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:24.148712 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:24.148774 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:24.148933 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:24.148934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:24.149140 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:24.149218 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:24.149219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.149969 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.149970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:24.150586 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:24.150588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.151030 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.151031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:24.151079 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:24.151543 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:24.151548 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:24.153619 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":117,"threads_started":1}
W20260812 00:07:24.153779 25370 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:24.148629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:59841 (local address 127.7.85.254:33189)
0812 00:07:24.148796 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:24.148798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:24.148823 (+ 25us) server_negotiation.cc:407] Connection header received
0812 00:07:24.148846 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:24.148847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:24.148859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:24.148917 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:24.149313 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.149889 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.150690 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.150921 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.150987 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:24.153708 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:59841: BlockingRecv error: recv got EOF from 127.7.85.193:59841 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
Aug 12 00:07:24 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:24 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:24.313347 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:24.308105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:57469)
0812 00:07:24.308256 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:24.308270 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:24.308336 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:24.308499 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:24.308500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:24.308706 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:24.308779 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:24.308781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.309669 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.309670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:24.310236 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:24.310237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.310633 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.310634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:24.310701 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:24.311169 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:24.311175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:24.313274 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
W20260812 00:07:24.313413 25370 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:24.308204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:57469 (local address 127.7.85.254:33189)
0812 00:07:24.308303 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:24.308305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:24.308323 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:07:24.308401 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:24.308402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:24.308414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:24.308478 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:24.308994 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.309581 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.310311 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.310543 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.310605 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:24.313342 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:57469: BlockingRecv error: recv got EOF from 127.7.85.194:57469 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 12 00:07:24 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:24 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:24.463284 25376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:24.458494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:39235)
0812 00:07:24.458714 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:24.458726 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:24.458805 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:24.458962 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:24.458963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:24.459167 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:24.459216 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:24.459217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.459901 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.459902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:24.460457 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:24.460459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.460803 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.460804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:24.460857 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:24.461373 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:24.461378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:24.463164 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":103,"threads_started":1}
W20260812 00:07:24.463313 25370 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:24.458590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:39235 (local address 127.7.85.254:33189)
0812 00:07:24.458696 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:24.458699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:24.458783 (+ 84us) server_negotiation.cc:407] Connection header received
0812 00:07:24.458866 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:24.458867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:24.458882 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:24.458952 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:24.459296 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.459818 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.460546 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:24.460731 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:24.460811 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:24.463243 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:39235: BlockingRecv error: recv got EOF from 127.7.85.195:39235 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 00:07:25 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:25 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:25.159762 25379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:25.154325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:46803)
0812 00:07:25.154559 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:25.154575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:25.154634 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:25.154772 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:25.154773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:25.154958 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:25.155012 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:25.155013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.155766 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.155767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:25.156440 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:25.156444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.156835 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.156836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:25.156898 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:25.157525 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:25.157531 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:25.159695 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":181,"threads_started":1}
W20260812 00:07:25.159936 25382 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:25.154443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:46803 (local address 127.7.85.254:33189)
0812 00:07:25.154579 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:25.154581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:25.154618 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:07:25.154686 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:25.154687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:25.154698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:25.154769 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:25.155122 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.155651 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.156539 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.156738 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.156806 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:25.159825 (+ 3019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:46803: BlockingRecv error: recv got EOF from 127.7.85.193:46803 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
Aug 12 00:07:25 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:25 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:25.319312 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:25.313934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:44089)
0812 00:07:25.314132 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:25.314149 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:25.314227 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:25.314430 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:25.314432 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:25.314643 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:25.314693 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:25.314695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.315442 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.315443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:25.316054 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:25.316056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.316508 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.316510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:25.316578 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:25.317212 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:25.317217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:25.319185 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
W20260812 00:07:25.319355 25382 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:25.314115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:44089 (local address 127.7.85.254:33189)
0812 00:07:25.314226 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:25.314228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:25.314250 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:07:25.314308 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:25.314309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:25.314321 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:25.314385 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:25.314803 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.315341 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.316154 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.316409 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.316483 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:25.319282 (+ 2799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:44089: BlockingRecv error: recv got EOF from 127.7.85.194:44089 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 00:07:25 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:25 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:25.469664 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:25.464074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:41291)
0812 00:07:25.464383 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:25.464406 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:25.464526 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:25.464754 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:25.464755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:25.464976 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:25.465030 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:25.465032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.465878 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.465880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:25.466457 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:25.466458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.466859 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.466860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:25.466929 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:25.467522 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:25.467528 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:25.469600 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":182,"threads_started":1}
W20260812 00:07:25.469887 25382 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:25.464258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:41291 (local address 127.7.85.254:33189)
0812 00:07:25.464359 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:25.464361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:25.464496 (+ 135us) server_negotiation.cc:407] Connection header received
0812 00:07:25.464626 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:25.464627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:25.464649 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:25.464730 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:25.465120 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.465770 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.466546 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:25.466764 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:25.466857 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:25.469811 (+ 2954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:41291: BlockingRecv error: recv got EOF from 127.7.85.195:41291 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 00:07:26 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:26 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:26.166133 25388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:26.160364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:60263)
0812 00:07:26.160534 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:26.160547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:26.160605 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:26.160917 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:26.160920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:26.161353 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:26.161417 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:26.161422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.162145 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.162146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:26.162678 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:26.162679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.163063 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.163064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:26.163119 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:26.163640 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:26.163645 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:26.166028 (+ 2383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
W20260812 00:07:26.166204 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:26.160451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:60263 (local address 127.7.85.254:33189)
0812 00:07:26.160714 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:26.160716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:26.160726 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:07:26.160754 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:26.160754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:26.160769 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:26.160877 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:26.161524 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.162054 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.162760 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.162963 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.163063 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:26.166128 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:60263: BlockingRecv error: recv got EOF from 127.7.85.193:60263 (error 108)
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":160,"threads_started":1}
Aug 12 00:07:26 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:26 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:26.325378 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:26.319908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:54723)
0812 00:07:26.320081 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:26.320093 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:26.320148 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:26.320472 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:26.320473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:26.320663 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:26.320712 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:26.320714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.321506 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.321507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:26.322068 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:26.322070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.322481 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.322483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:26.322530 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:26.323147 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:26.323152 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:26.325257 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
W20260812 00:07:26.325417 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:26.320019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:54723 (local address 127.7.85.254:33189)
0812 00:07:26.320084 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:26.320086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:26.320151 (+ 65us) server_negotiation.cc:407] Connection header received
0812 00:07:26.320291 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:26.320292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:26.320306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:26.320365 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:26.320814 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.321384 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.322161 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.322401 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.322467 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:26.325349 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:54723: BlockingRecv error: recv got EOF from 127.7.85.194:54723 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 00:07:26 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:26 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.195@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:26.475142 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:26.470242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.195:60007)
0812 00:07:26.470494 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:26.470508 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:26.470602 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:26.470764 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:26.470765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:26.470968 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:26.471015 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:26.471017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.471687 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.471688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:26.472213 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:26.472214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.472566 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.472567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:26.472624 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:26.473155 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:26.473160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:26.475076 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":99,"threads_started":1}
W20260812 00:07:26.475314 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:26.470378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:60007 (local address 127.7.85.254:33189)
0812 00:07:26.470484 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:26.470487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:26.470583 (+ 96us) server_negotiation.cc:407] Connection header received
0812 00:07:26.470668 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:26.470669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:26.470691 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:26.470755 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:26.471095 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.471605 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.472301 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:26.472493 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:26.472567 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:26.475225 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.195:60007: BlockingRecv error: recv got EOF from 127.7.85.195:60007 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 00:07:27 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:27 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.193@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:27.175542 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.166841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.193:39119)
0812 00:07:27.167014 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:27.167026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:27.167087 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:27.167312 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:27.167314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:27.167504 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:27.167557 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.167559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.168280 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.168281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.168848 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.168849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.169350 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.169351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.169423 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.173259 (+ 3836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:27.173265 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:27.175448 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
W20260812 00:07:27.175648 25394 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:27.167001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:39119 (local address 127.7.85.254:33189)
0812 00:07:27.167152 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:27.167154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:27.167163 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:27.167190 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:27.167191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:27.167203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:27.167270 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:27.167677 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.168188 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.168943 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.169160 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.169348 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.175570 (+ 6222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.193:39119: BlockingRecv error: recv got EOF from 127.7.85.193:39119 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
Aug 12 00:07:27 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:07:27 dist-test-slave-z8x9 krb5kdc[23479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.85.194@KRBTEST.COM for oryx/127.7.85.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:07:27.331024 25398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.325998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33189 (local address 127.7.85.194:50559)
0812 00:07:27.326150 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:27.326162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:27.326248 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:27.326387 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:27.326388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:27.326591 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:27.326638 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.326640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.327320 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.327322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.327855 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.327857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.328213 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.328214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.328272 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.328830 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:27.328835 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:27.330963 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.85.254:33189: Server oryx/127.7.85.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
W20260812 00:07:27.331220 25394 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:27.326051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:50559 (local address 127.7.85.254:33189)
0812 00:07:27.326108 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:27.326110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:27.326232 (+ 122us) server_negotiation.cc:407] Connection header received
0812 00:07:27.326317 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:27.326318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:27.326331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:27.326382 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:27.326717 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.327226 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.327941 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.328137 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.328215 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.331153 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.85.194:50559: BlockingRecv error: recv got EOF from 127.7.85.194:50559 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:07:27.380497 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 24532
I20260812 00:07:27.397382 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 24803
I20260812 00:07:27.408289 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 25037
I20260812 00:07:27.420836 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 24416
2026-08-12T00:07:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18177 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T00:07:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:27Z Disabled control of system clock
I20260812 00:07:27.455368 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:34951
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 00:07:27.543430 25404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:27.543668 25404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:27.543690 25404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:27.544941 25404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:27.544991 25404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:27.545003 25404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:27.545017 25404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:27.545028 25404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:27.546520 25404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:34951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:27.546808 25404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:27.547125 25404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:07:27.552897 25404 server_base.cc:1034] running on GCE node
W20260812 00:07:27.553068 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:27.553084 25418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:27.553084 25417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:27.553608 25404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:27.553980 25404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:27.555132 25404 hybrid_clock.cc:648] HybridClock initialized: now 1786493247555112 us; error 38 us; skew 500 ppm
I20260812 00:07:27.556576 25404 webserver.cc:460] Webserver started at http://127.7.85.254:38381/ using document root <none> and password file <none>
I20260812 00:07:27.556861 25404 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:27.556916 25404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:27.557083 25404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:27.558619 25404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "a88a6092e8384a9caa046bef2d6d365c"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "dd50164f8ed3007bf40c2339577b4142"
server_key_iv: "300e0fc6efc5302f00ebae2c14028284"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.559015 25404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "a88a6092e8384a9caa046bef2d6d365c"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "dd50164f8ed3007bf40c2339577b4142"
server_key_iv: "300e0fc6efc5302f00ebae2c14028284"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.560384 25404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:07:27.561255 25436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.561494 25404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:27.561606 25404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "a88a6092e8384a9caa046bef2d6d365c"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "dd50164f8ed3007bf40c2339577b4142"
server_key_iv: "300e0fc6efc5302f00ebae2c14028284"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.561719 25404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:27.581575 25404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:27.581944 25404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:27.582103 25404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:27.586712 25404 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:34951
I20260812 00:07:27.586799 25530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:34951 every 8 connection(s)
I20260812 00:07:27.587142 25404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:07:27.587954 25532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:27.590680 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 25404
I20260812 00:07:27.590772 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:07:27.590989 7511 external_mini_cluster.cc:1428] Setting key f77a3c65a4f92a51de2609137d516b68
I20260812 00:07:27.590977 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c: Bootstrap starting.
I20260812 00:07:27.591799 25532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:27.592152 25532 log.cc:824] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:27.592969 25532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c: No bootstrap required, opened a new log
I20260812 00:07:27.594816 25532 raft_consensus.cc:348] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88a6092e8384a9caa046bef2d6d365c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34951 } }
I20260812 00:07:27.594955 25532 raft_consensus.cc:374] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:27.594978 25532 raft_consensus.cc:729] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88a6092e8384a9caa046bef2d6d365c, State: Initialized, Role: FOLLOWER
I20260812 00:07:27.595113 25532 consensus_queue.cc:260] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88a6092e8384a9caa046bef2d6d365c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34951 } }
I20260812 00:07:27.595208 25532 raft_consensus.cc:388] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:07:27.595252 25532 raft_consensus.cc:482] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:07:27.595332 25532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:27.596274 25532 raft_consensus.cc:504] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88a6092e8384a9caa046bef2d6d365c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34951 } }
I20260812 00:07:27.596400 25532 leader_election.cc:302] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a88a6092e8384a9caa046bef2d6d365c; no voters:
I20260812 00:07:27.596580 25532 leader_election.cc:288] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:07:27.596611 25549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:27.596791 25549 raft_consensus.cc:686] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [term 1 LEADER]: Becoming Leader. State: Replica: a88a6092e8384a9caa046bef2d6d365c, State: Running, Role: LEADER
I20260812 00:07:27.596953 25532 sys_catalog.cc:564] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:07:27.596953 25549 consensus_queue.cc:237] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88a6092e8384a9caa046bef2d6d365c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34951 } }
I20260812 00:07:27.597621 25550 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a88a6092e8384a9caa046bef2d6d365c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88a6092e8384a9caa046bef2d6d365c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34951 } } }
I20260812 00:07:27.597661 25551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a88a6092e8384a9caa046bef2d6d365c. Latest consensus state: current_term: 1 leader_uuid: "a88a6092e8384a9caa046bef2d6d365c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a88a6092e8384a9caa046bef2d6d365c" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34951 } } }
I20260812 00:07:27.597744 25550 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:27.597757 25551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:27.598150 25562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:07:27.598215 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.591596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.7.85.254:34951)
0812 00:07:27.591844 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:27.591853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:27.591889 (+ 36us) server_negotiation.cc:407] Connection header received
0812 00:07:27.592496 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:27.592503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:27.592564 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:27.592733 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:27.592945 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.594392 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.595309 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.595782 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.595925 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.596909 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:27.596913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:27.596913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:27.597123 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:27.597196 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:27.597214 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:27.597274 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:27.597507 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 00:07:27.597596 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260812 00:07:27.598697 25562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:27.600560 25562 catalog_manager.cc:1347] Generated new cluster ID: 317100aeff2347a4bd4639b8211efef8
I20260812 00:07:27.600623 25562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:07:27.609109 25562 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:07:27.609840 25562 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:27.613854 25562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a88a6092e8384a9caa046bef2d6d365c: Generated new TSK 0
I20260812 00:07:27.614046 25562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:07:27.623327 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34951
--builtin_ntp_servers=127.7.85.212:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 00:07:27.714721 25579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:27.714955 25579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:27.714979 25579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:27.716174 25579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:27.716224 25579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:27.716251 25579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:07:27.718562 25579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:27.718989 25579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:27.719419 25579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:27.727149 25586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:27.727150 25595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:27.727149 25592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:27.727528 25579 server_base.cc:1034] running on GCE node
I20260812 00:07:27.727764 25579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:27.728055 25579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:27.729249 25579 hybrid_clock.cc:648] HybridClock initialized: now 1786493247729219 us; error 40 us; skew 500 ppm
I20260812 00:07:27.730906 25579 webserver.cc:460] Webserver started at http://127.7.85.193:33791/ using document root <none> and password file <none>
I20260812 00:07:27.731364 25579 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:27.731452 25579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:27.731598 25579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:27.733330 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "be53c95bc51d49f0ae747748f8062925"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "83ce43827bb3604392134bae28723e58"
server_key_iv: "46117b7b56d1187a55ea79e05b16c9a7"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.733824 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "be53c95bc51d49f0ae747748f8062925"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "83ce43827bb3604392134bae28723e58"
server_key_iv: "46117b7b56d1187a55ea79e05b16c9a7"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.735937 25579 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:27.737071 25603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.737337 25579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:27.737450 25579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "be53c95bc51d49f0ae747748f8062925"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "83ce43827bb3604392134bae28723e58"
server_key_iv: "46117b7b56d1187a55ea79e05b16c9a7"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.737524 25579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:27.752238 25579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:27.752578 25579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:27.752696 25579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:27.752944 25579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:27.753391 25579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:27.753433 25579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.753458 25579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:27.753471 25579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.759562 25579 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:38679
I20260812 00:07:27.759611 25795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:38679 every 8 connection(s)
I20260812 00:07:27.760022 25579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:07:27.765859 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.760868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:57007 (local address 127.7.85.254:34951)
0812 00:07:27.760936 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:27.760938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:27.761274 (+ 336us) server_negotiation.cc:407] Connection header received
0812 00:07:27.761783 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:27.761785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:27.761795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:27.761860 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:27.762579 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.763109 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.764136 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.764365 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.764475 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.765546 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:27.765547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:27.765548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:27.765554 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:27.765571 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:27.765576 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:27.765627 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:27.765741 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 00:07:27.765770 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 00:07:27.766113 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.760866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34951 (local address 127.7.85.193:57007)
0812 00:07:27.761075 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:27.761110 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:27.761675 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:27.761926 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:27.761929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:27.761938 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:27.762455 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.762459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.763216 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.763217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.764057 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.764059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.764509 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.764510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.764643 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.765384 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:07:27.765385 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:27.765415 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:07:27.765439 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:27.765627 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:27.765634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:27.765638 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:27.765737 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:07:27.765848 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260812 00:07:27.766701 25796 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34951
I20260812 00:07:27.766798 25796 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:27.766963 25796 heartbeater.cc:511] Master 127.7.85.254:34951 requested a full tablet report, sending...
I20260812 00:07:27.767524 25470 ts_manager.cc:194] Registered new tserver with Master: be53c95bc51d49f0ae747748f8062925 (127.7.85.193:38679)
I20260812 00:07:27.768115 25470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.193:57007
I20260812 00:07:27.770123 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 25579
I20260812 00:07:27.770211 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:07:27.770380 7511 external_mini_cluster.cc:1428] Setting key a9e469a851994a69b839618402581472
I20260812 00:07:27.771431 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34951
--builtin_ntp_servers=127.7.85.212:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 00:07:27.865516 25800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:27.865782 25800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:27.865808 25800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:27.867162 25800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:27.867216 25800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:27.867245 25800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:07:27.868793 25800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:27.869089 25800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:27.869597 25800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:27.875314 25808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:27.875404 25800 server_base.cc:1034] running on GCE node
W20260812 00:07:27.875355 25813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:27.875367 25811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:27.875909 25800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:27.876137 25800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:27.877234 25800 hybrid_clock.cc:648] HybridClock initialized: now 1786493247877201 us; error 51 us; skew 500 ppm
I20260812 00:07:27.878706 25800 webserver.cc:460] Webserver started at http://127.7.85.194:38551/ using document root <none> and password file <none>
I20260812 00:07:27.879058 25800 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:27.879276 25800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:27.879452 25800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:27.880720 25800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "a7a1b6d7177844fcb881120671eb01b7"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "258351a3eb26c892924214ad52254048"
server_key_iv: "8ac8473148875eb7cf3ba134d89c2f23"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.881086 25800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "a7a1b6d7177844fcb881120671eb01b7"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "258351a3eb26c892924214ad52254048"
server_key_iv: "8ac8473148875eb7cf3ba134d89c2f23"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.882552 25800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:27.883442 25821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.883745 25800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:27.884002 25800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "a7a1b6d7177844fcb881120671eb01b7"
format_stamp: "Formatted at 2026-08-12 00:07:27 on dist-test-slave-z8x9"
server_key: "258351a3eb26c892924214ad52254048"
server_key_iv: "8ac8473148875eb7cf3ba134d89c2f23"
server_key_version: "encryptionkey@0"
I20260812 00:07:27.884111 25800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:27.906745 25800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:27.907100 25800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:27.907274 25800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:27.907534 25800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:27.907905 25800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:27.907963 25800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.908008 25800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:27.908037 25800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:27.914108 25800 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:43011
I20260812 00:07:27.914187 26021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:43011 every 8 connection(s)
I20260812 00:07:27.914530 25800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:07:27.918046 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 25800
I20260812 00:07:27.918141 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:07:27.918344 7511 external_mini_cluster.cc:1428] Setting key 0fa97b89c10ce2b8b8683e87780f6a62
I20260812 00:07:27.919634 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34951
--builtin_ntp_servers=127.7.85.212:39195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 00:07:27.920656 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.915325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:33323 (local address 127.7.85.254:34951)
0812 00:07:27.915386 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:27.915388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:27.915615 (+ 227us) server_negotiation.cc:407] Connection header received
0812 00:07:27.916091 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:27.916092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:27.916101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:27.916167 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:27.916819 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.917413 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.918913 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.919226 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.919310 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.920396 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:27.920397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:27.920398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:27.920402 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:27.920418 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:27.920422 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:27.920477 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:27.920572 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 00:07:27.920596 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:07:27.920960 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:27.915292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34951 (local address 127.7.85.194:33323)
0812 00:07:27.915493 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:27.915535 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:27.916009 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:27.916217 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:27.916221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:27.916230 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:27.916749 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.916753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.917531 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.917533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.918746 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:27.918749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:27.919359 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:27.919360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:27.919532 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:27.920217 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:07:27.920217 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:27.920258 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:07:27.920289 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:27.920494 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:27.920500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:27.920503 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:07:27.920573 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 00:07:27.920685 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260812 00:07:27.921660 26022 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34951
I20260812 00:07:27.921777 26022 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:27.921988 26022 heartbeater.cc:511] Master 127.7.85.254:34951 requested a full tablet report, sending...
I20260812 00:07:27.922399 25470 ts_manager.cc:194] Registered new tserver with Master: a7a1b6d7177844fcb881120671eb01b7 (127.7.85.194:43011)
I20260812 00:07:27.922839 25470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.194:33323
W20260812 00:07:28.007649 26027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:28.007884 26027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:28.007906 26027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:28.009115 26027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:28.009189 26027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:28.009222 26027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:28.010687 26027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:39195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34951
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:28.010942 26027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:28.011264 26027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:28.016533 26040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.016672 26027 server_base.cc:1034] running on GCE node
W20260812 00:07:28.016546 26035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.016551 26037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.017022 26027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:28.017379 26027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:28.018541 26027 hybrid_clock.cc:648] HybridClock initialized: now 1786493248018528 us; error 31 us; skew 500 ppm
I20260812 00:07:28.019892 26027 webserver.cc:460] Webserver started at http://127.7.85.195:39259/ using document root <none> and password file <none>
I20260812 00:07:28.020164 26027 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:28.020238 26027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:28.020391 26027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:28.021636 26027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "0b66afd3d0c049d38bfb2b05439a3dcb"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "0acfa1f6d49f3e221f9a90d0c2c20a39"
server_key_iv: "1dd2c326b9130d8368572920b9e97248"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.022063 26027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "0b66afd3d0c049d38bfb2b05439a3dcb"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "0acfa1f6d49f3e221f9a90d0c2c20a39"
server_key_iv: "1dd2c326b9130d8368572920b9e97248"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.023440 26027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:07:28.024240 26049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.024433 26027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:28.024535 26027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "0b66afd3d0c049d38bfb2b05439a3dcb"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "0acfa1f6d49f3e221f9a90d0c2c20a39"
server_key_iv: "1dd2c326b9130d8368572920b9e97248"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.024619 26027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:28.039633 26027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:28.040001 26027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:28.040182 26027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:28.040464 26027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:28.040823 26027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:28.040877 26027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.040920 26027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:28.040958 26027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.047523 26027 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34349
I20260812 00:07:28.047570 26260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34349 every 8 connection(s)
I20260812 00:07:28.047910 26027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:07:28.053651 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.048702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:46675 (local address 127.7.85.254:34951)
0812 00:07:28.048769 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.048771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.049010 (+ 239us) server_negotiation.cc:407] Connection header received
0812 00:07:28.049643 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.049645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.049659 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.049706 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.050426 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:28.050944 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:28.051943 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:28.052151 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:28.052234 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:28.053307 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:28.053309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:28.053310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:28.053316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:28.053334 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:28.053339 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:28.053414 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:28.053524 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 00:07:28.053556 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 00:07:28.053922 26264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.048664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34951 (local address 127.7.85.195:46675)
0812 00:07:28.048890 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:28.048925 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:28.049523 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:28.049788 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:28.049792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:28.049801 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:28.050333 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:28.050338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:28.051048 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:28.051049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:28.051843 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:28.051845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:28.052250 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:28.052251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:28.052386 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:28.053020 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:07:28.053020 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:28.053052 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:07:28.053076 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:28.053426 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:28.053433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:28.053437 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:28.053526 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 00:07:28.053639 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
I20260812 00:07:28.054492 26261 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34951
I20260812 00:07:28.054605 26261 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:28.054826 26261 heartbeater.cc:511] Master 127.7.85.254:34951 requested a full tablet report, sending...
I20260812 00:07:28.055243 25470 ts_manager.cc:194] Registered new tserver with Master: 0b66afd3d0c049d38bfb2b05439a3dcb (127.7.85.195:34349)
I20260812 00:07:28.055617 25470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.195:46675
I20260812 00:07:28.056753 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 26027
I20260812 00:07:28.056856 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:07:28.057078 7511 external_mini_cluster.cc:1428] Setting key 20e58bdcfeb5140835b0bafae8e82013
I20260812 00:07:28.058094 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 00:07:28.060019 25545 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:28.059424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.7.85.254:34951)
0812 00:07:28.059491 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.059494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.059614 (+ 120us) server_negotiation.cc:407] Connection header received
0812 00:07:28.059691 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.059692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.059707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.059807 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.059952 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55892: BlockingRecv error: recv got EOF from 127.0.0.1:55892 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:07:28.060155 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 25579
I20260812 00:07:28.086561 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 25800
I20260812 00:07:28.108949 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 26027
I20260812 00:07:28.124370 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 25404
2026-08-12T00:07:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (709 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T00:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:28Z Disabled control of system clock
I20260812 00:07:28.165831 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:46211
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:46211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 00:07:28.258261 26289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:28.258598 26289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:28.258808 26289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:28.260775 26289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:28.260828 26289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:28.260843 26289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:28.260859 26289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:28.260874 26289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:28.262617 26289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:46211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:46211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:28.262884 26289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:28.263186 26289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:28.268725 26302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.268739 26305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.268764 26300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.268772 26289 server_base.cc:1034] running on GCE node
I20260812 00:07:28.269320 26289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:28.269731 26289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:28.270931 26289 hybrid_clock.cc:648] HybridClock initialized: now 1786493248270895 us; error 53 us; skew 500 ppm
I20260812 00:07:28.272440 26289 webserver.cc:460] Webserver started at http://127.7.85.254:42127/ using document root <none> and password file <none>
I20260812 00:07:28.272694 26289 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:28.272771 26289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:28.272924 26289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:28.274212 26289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "5e90f48d369d47ed9d49a0f73783faef"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "c4a151704c932616ae5c4d041a705da8"
server_key_iv: "292b7ec316014a68d5d870ffff54c4ff"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.274591 26289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "5e90f48d369d47ed9d49a0f73783faef"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "c4a151704c932616ae5c4d041a705da8"
server_key_iv: "292b7ec316014a68d5d870ffff54c4ff"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.276003 26289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:07:28.277021 26316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.277383 26289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:28.277520 26289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "5e90f48d369d47ed9d49a0f73783faef"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "c4a151704c932616ae5c4d041a705da8"
server_key_iv: "292b7ec316014a68d5d870ffff54c4ff"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.277606 26289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:28.281349 26289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:28.281654 26289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:28.281803 26289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:28.286553 26289 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:46211
I20260812 00:07:28.286602 26419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:46211 every 8 connection(s)
I20260812 00:07:28.286988 26289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:07:28.287722 26421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:28.290781 26421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef: Bootstrap starting.
I20260812 00:07:28.291462 26421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:28.291792 26421 log.cc:824] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:28.292667 26421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef: No bootstrap required, opened a new log
I20260812 00:07:28.292999 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 26289
I20260812 00:07:28.293114 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:07:28.293368 7511 external_mini_cluster.cc:1428] Setting key ee8b7b5a66b90c3c8476672e305a7782
I20260812 00:07:28.295197 26421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e90f48d369d47ed9d49a0f73783faef" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46211 } }
I20260812 00:07:28.295374 26421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:28.295399 26421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e90f48d369d47ed9d49a0f73783faef, State: Initialized, Role: FOLLOWER
I20260812 00:07:28.295558 26421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e90f48d369d47ed9d49a0f73783faef" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46211 } }
I20260812 00:07:28.295660 26421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:07:28.295698 26421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:07:28.295799 26421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:28.296706 26421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e90f48d369d47ed9d49a0f73783faef" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46211 } }
I20260812 00:07:28.296856 26421 leader_election.cc:302] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e90f48d369d47ed9d49a0f73783faef; no voters:
I20260812 00:07:28.297006 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.294252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.7.85.254:46211)
0812 00:07:28.294584 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.294595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.294634 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:07:28.295204 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.295216 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.295300 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.295505 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.296414 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:28.296418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:28.296418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:28.296422 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:28.296474 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:28.296486 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:28.296552 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:28.296654 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 00:07:28.296741 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":152,"threads_started":1}
I20260812 00:07:28.297107 26421 leader_election.cc:288] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:07:28.297456 26421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:07:28.297674 26429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:28.297837 26429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [term 1 LEADER]: Becoming Leader. State: Replica: 5e90f48d369d47ed9d49a0f73783faef, State: Running, Role: LEADER
I20260812 00:07:28.297981 26429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e90f48d369d47ed9d49a0f73783faef" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46211 } }
I20260812 00:07:28.298499 26429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e90f48d369d47ed9d49a0f73783faef. Latest consensus state: current_term: 1 leader_uuid: "5e90f48d369d47ed9d49a0f73783faef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e90f48d369d47ed9d49a0f73783faef" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46211 } } }
I20260812 00:07:28.298528 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e90f48d369d47ed9d49a0f73783faef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e90f48d369d47ed9d49a0f73783faef" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 46211 } } }
I20260812 00:07:28.298691 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:28.298677 26429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:28.299176 26446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:07:28.299738 26446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:28.301409 26446 catalog_manager.cc:1347] Generated new cluster ID: 4b99bf5137ae4a9a83c326f0fd1022ba
I20260812 00:07:28.301477 26446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:07:28.323352 26446 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:07:28.323779 26446 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:28.334424 26446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e90f48d369d47ed9d49a0f73783faef: Generated new TSK 0
I20260812 00:07:28.334889 26446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:07:28.347921 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46211
--builtin_ntp_servers=127.7.85.212:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 00:07:28.470660 26462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:28.470897 26462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:28.470922 26462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:28.472146 26462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:28.472203 26462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:28.472234 26462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:07:28.473763 26462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:28.474037 26462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:28.474361 26462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:28.480715 26472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.481112 26470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.481145 26468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.481263 26462 server_base.cc:1034] running on GCE node
I20260812 00:07:28.481556 26462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:28.481853 26462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:28.483098 26462 hybrid_clock.cc:648] HybridClock initialized: now 1786493248483062 us; error 28 us; skew 500 ppm
I20260812 00:07:28.484709 26462 webserver.cc:460] Webserver started at http://127.7.85.193:33553/ using document root <none> and password file <none>
I20260812 00:07:28.484978 26462 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:28.485028 26462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:28.485216 26462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:28.486537 26462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "853b312a165c402e80915f14862e3cd8"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "10dda0083df58c357a00a654cdc9a8fc"
server_key_iv: "70e6a2fe55dc88cce17a35239546509e"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.486966 26462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "853b312a165c402e80915f14862e3cd8"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "10dda0083df58c357a00a654cdc9a8fc"
server_key_iv: "70e6a2fe55dc88cce17a35239546509e"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.488452 26462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:07:28.489823 26483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.490020 26462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:28.490109 26462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "853b312a165c402e80915f14862e3cd8"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "10dda0083df58c357a00a654cdc9a8fc"
server_key_iv: "70e6a2fe55dc88cce17a35239546509e"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.490170 26462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:28.494048 26462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:28.494313 26462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:28.494427 26462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:28.494611 26462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:28.494890 26462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:28.494921 26462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.494942 26462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:28.494956 26462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.501808 26462 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:35287
I20260812 00:07:28.501850 26677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:35287 every 8 connection(s)
I20260812 00:07:28.502215 26462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:07:28.505914 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 26462
I20260812 00:07:28.506032 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:07:28.506171 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.503157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:45401 (local address 127.7.85.254:46211)
0812 00:07:28.503247 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.503249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.503467 (+ 218us) server_negotiation.cc:407] Connection header received
0812 00:07:28.504179 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.504181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.504191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.504244 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.505831 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:28.505832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:28.505833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:28.505837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:28.505854 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:28.505858 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:28.505918 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:28.506060 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 00:07:28.506079 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:07:28.506283 7511 external_mini_cluster.cc:1428] Setting key 3af78a2217dfa61f502a8c7ee7e382d6
I20260812 00:07:28.506439 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.503144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46211 (local address 127.7.85.193:45401)
0812 00:07:28.503348 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:28.503384 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:28.504041 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:28.504342 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:28.504346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:28.504354 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:28.505622 (+ 1268us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:07:28.505622 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:28.505653 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:07:28.505703 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:28.505934 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:28.505941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:28.505945 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:28.506044 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:07:28.506117 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260812 00:07:28.507098 26678 heartbeater.cc:347] Connected to a master server at 127.7.85.254:46211
I20260812 00:07:28.507216 26678 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:28.507313 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46211
--builtin_ntp_servers=127.7.85.212:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 00:07:28.507388 26678 heartbeater.cc:511] Master 127.7.85.254:46211 requested a full tablet report, sending...
I20260812 00:07:28.507902 26354 ts_manager.cc:194] Registered new tserver with Master: 853b312a165c402e80915f14862e3cd8 (127.7.85.193:35287)
W20260812 00:07:28.594089 26682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:28.594347 26682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:28.594372 26682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:28.595531 26682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:28.595571 26682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:28.595597 26682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:07:28.596992 26682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:28.597282 26682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:28.597617 26682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:28.603147 26695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.603229 26702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.603330 26696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.603857 26682 server_base.cc:1034] running on GCE node
I20260812 00:07:28.604430 26682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:28.604707 26682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:28.605895 26682 hybrid_clock.cc:648] HybridClock initialized: now 1786493248605861 us; error 70 us; skew 500 ppm
I20260812 00:07:28.607331 26682 webserver.cc:460] Webserver started at http://127.7.85.194:39427/ using document root <none> and password file <none>
I20260812 00:07:28.607615 26682 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:28.607685 26682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:28.607836 26682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:28.608951 26682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "0f97273dbf5b40abbcd60954e2eefa7a"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "6885071a332e38e955ea5c9b5ce660af"
server_key_iv: "bc0262ab85346806d8c6ec648ebf32d8"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.609386 26682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "0f97273dbf5b40abbcd60954e2eefa7a"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "6885071a332e38e955ea5c9b5ce660af"
server_key_iv: "bc0262ab85346806d8c6ec648ebf32d8"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.610888 26682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:28.611867 26713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.612093 26682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:28.612255 26682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "0f97273dbf5b40abbcd60954e2eefa7a"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "6885071a332e38e955ea5c9b5ce660af"
server_key_iv: "bc0262ab85346806d8c6ec648ebf32d8"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.612344 26682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:28.616360 26682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:28.616662 26682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:28.616816 26682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:28.617039 26682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:28.617443 26682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:28.617506 26682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.617552 26682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:28.617583 26682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.624326 26682 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35807
I20260812 00:07:28.624496 26931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35807 every 8 connection(s)
I20260812 00:07:28.624760 26682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:07:28.628021 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.625659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:37133 (local address 127.7.85.254:46211)
0812 00:07:28.625721 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.625724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.625944 (+ 220us) server_negotiation.cc:407] Connection header received
0812 00:07:28.626458 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.626459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.626466 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.626526 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.627751 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:28.627753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:28.627753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:28.627758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:28.627776 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:28.627780 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:28.627836 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:28.627931 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 00:07:28.627947 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 00:07:28.628248 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.625607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46211 (local address 127.7.85.194:37133)
0812 00:07:28.625805 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:28.625856 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:28.626352 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:28.626604 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:28.626607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:28.626615 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:28.627597 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:07:28.627598 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:28.627627 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:07:28.627655 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:28.627834 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:28.627840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:28.627843 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:07:28.627928 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 00:07:28.627992 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260812 00:07:28.628783 26932 heartbeater.cc:347] Connected to a master server at 127.7.85.254:46211
I20260812 00:07:28.628881 26932 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:28.629042 26932 heartbeater.cc:511] Master 127.7.85.254:46211 requested a full tablet report, sending...
I20260812 00:07:28.629490 26354 ts_manager.cc:194] Registered new tserver with Master: 0f97273dbf5b40abbcd60954e2eefa7a (127.7.85.194:35807)
I20260812 00:07:28.633658 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 26682
I20260812 00:07:28.633769 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:07:28.633975 7511 external_mini_cluster.cc:1428] Setting key 42af2d30190412c37fc076b176cc4a85
I20260812 00:07:28.635244 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46211
--builtin_ntp_servers=127.7.85.212:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 00:07:28.724475 26938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:28.724696 26938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:28.724838 26938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:28.726197 26938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:28.726244 26938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:28.726274 26938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:28.727715 26938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:46211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:28.727932 26938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:28.728214 26938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:28.733525 26947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:28.733566 26948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.733769 26938 server_base.cc:1034] running on GCE node
W20260812 00:07:28.733776 26950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:28.734090 26938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:28.734297 26938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:28.735436 26938 hybrid_clock.cc:648] HybridClock initialized: now 1786493248735417 us; error 41 us; skew 500 ppm
I20260812 00:07:28.736785 26938 webserver.cc:460] Webserver started at http://127.7.85.195:34059/ using document root <none> and password file <none>
I20260812 00:07:28.737039 26938 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:28.737108 26938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:28.737295 26938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:28.738379 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "990b3150cf474f9b8310426a53c50a48"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "1cfa4b9be521e9ab20dc964eb8bc5860"
server_key_iv: "30584f4e4d809e72729e76935c28f026"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.738741 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "990b3150cf474f9b8310426a53c50a48"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "1cfa4b9be521e9ab20dc964eb8bc5860"
server_key_iv: "30584f4e4d809e72729e76935c28f026"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.740134 26938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:07:28.741252 26958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.741503 26938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:28.741604 26938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "990b3150cf474f9b8310426a53c50a48"
format_stamp: "Formatted at 2026-08-12 00:07:28 on dist-test-slave-z8x9"
server_key: "1cfa4b9be521e9ab20dc964eb8bc5860"
server_key_iv: "30584f4e4d809e72729e76935c28f026"
server_key_version: "encryptionkey@0"
I20260812 00:07:28.741714 26938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:28.745244 26938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:28.745556 26938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:28.745702 26938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:28.745911 26938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:28.746230 26938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:28.746282 26938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.746331 26938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:28.746359 26938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:28.752266 26938 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:35941
I20260812 00:07:28.752353 27168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:35941 every 8 connection(s)
I20260812 00:07:28.752687 26938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:07:28.755949 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.753502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:37511 (local address 127.7.85.254:46211)
0812 00:07:28.753581 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.753583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.753825 (+ 242us) server_negotiation.cc:407] Connection header received
0812 00:07:28.754336 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.754337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.754343 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.754403 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.755696 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:28.755698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:28.755698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:07:28.755702 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:28.755718 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:07:28.755722 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:28.755781 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:28.755876 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 00:07:28.755890 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:07:28.756116 27171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:28.753495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:46211 (local address 127.7.85.195:37511)
0812 00:07:28.753695 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:28.753736 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:28.754215 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:28.754463 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:28.754466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:28.754474 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:28.755520 (+ 1046us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:07:28.755520 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:28.755550 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:07:28.755579 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:28.755789 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:28.755794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:28.755796 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:07:28.755872 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:07:28.755922 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260812 00:07:28.756649 27169 heartbeater.cc:347] Connected to a master server at 127.7.85.254:46211
I20260812 00:07:28.756827 27169 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:28.756999 27169 heartbeater.cc:511] Master 127.7.85.254:46211 requested a full tablet report, sending...
I20260812 00:07:28.757444 26354 ts_manager.cc:194] Registered new tserver with Master: 990b3150cf474f9b8310426a53c50a48 (127.7.85.195:35941)
I20260812 00:07:28.761396 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 26938
I20260812 00:07:28.761492 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:07:28.761652 7511 external_mini_cluster.cc:1428] Setting key 36d061b1cf0bc3810af6bc649296724a
I20260812 00:07:28.762931 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 00:07:28.764906 26426 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:07:28.764468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40898 (local address 127.7.85.254:46211)
0812 00:07:28.764529 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:28.764530 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:28.764618 (+ 88us) server_negotiation.cc:407] Connection header received
0812 00:07:28.764672 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:28.764672 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:28.764677 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:28.764740 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:28.764857 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40898: BlockingRecv error: recv got EOF from 127.0.0.1:40898 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 00:07:28.765005 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 26462
I20260812 00:07:28.776342 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 26682
I20260812 00:07:28.798290 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 26938
I20260812 00:07:28.822345 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 26289
2026-08-12T00:07:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (695 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:07:28 dist-test-slave-z8x9 krb5kdc[27179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:07:28 dist-test-slave-z8x9 krb5kdc[27179](info): set up 2 sockets
Aug 12 00:07:28 dist-test-slave-z8x9 krb5kdc[27179](info): commencing operation
krb5kdc: starting...
W20260812 00:07:30.165375 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:07:30.304309 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40451
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:40451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:30.389796 27201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:30.390013 27201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:30.390035 27201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:30.391163 27201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:07:30.391206 27201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:30.391219 27201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:30.391233 27201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:07:30.391245 27201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:07:30.392781 27201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:40451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:30.393052 27201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:30.393311 27201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:30.398571 27211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.398581 27201 server_base.cc:1034] running on GCE node
W20260812 00:07:30.398602 27212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:30.398588 27217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.399140 27201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:30.399513 27201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:30.400681 27201 hybrid_clock.cc:648] HybridClock initialized: now 1786493250400663 us; error 31 us; skew 500 ppm
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:30.403415 27201 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:07:30.403887 27201 webserver.cc:460] Webserver started at http://127.7.85.254:39685/ using document root <none> and password file <none>
I20260812 00:07:30.404153 27201 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:30.404222 27201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:30.404376 27201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:30.405788 27201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "1bd58cede2ec45879b4ace39301a74a0"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "5a6d8f44799a63ab13026ad053acf432"
server_key_iv: "191724d544444f325045d84c35e9f024"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.406194 27201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "1bd58cede2ec45879b4ace39301a74a0"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "5a6d8f44799a63ab13026ad053acf432"
server_key_iv: "191724d544444f325045d84c35e9f024"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.407735 27201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:07:30.408645 27239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.408861 27201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:30.408973 27201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "1bd58cede2ec45879b4ace39301a74a0"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "5a6d8f44799a63ab13026ad053acf432"
server_key_iv: "191724d544444f325045d84c35e9f024"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.409050 27201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:30.419836 27201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:30.422770 27201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:30.422956 27201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:30.427405 27201 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:40451
I20260812 00:07:30.427438 27328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:40451 every 8 connection(s)
I20260812 00:07:30.427840 27201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:07:30.429003 27331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:30.430153 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 27201
I20260812 00:07:30.430269 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:07:30.430480 7511 external_mini_cluster.cc:1428] Setting key 7047a56e53b04981392840fa7986de18
I20260812 00:07:30.431819 27331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0: Bootstrap starting.
I20260812 00:07:30.432423 27331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:30.432785 27331 log.cc:824] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:30.433586 27331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0: No bootstrap required, opened a new log
I20260812 00:07:30.436060 27331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd58cede2ec45879b4ace39301a74a0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40451 } }
I20260812 00:07:30.436259 27331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:30.436292 27331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd58cede2ec45879b4ace39301a74a0, State: Initialized, Role: FOLLOWER
I20260812 00:07:30.436450 27331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd58cede2ec45879b4ace39301a74a0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40451 } }
I20260812 00:07:30.436550 27331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:07:30.436591 27331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:07:30.436642 27331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:30.437287 27331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd58cede2ec45879b4ace39301a74a0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40451 } }
I20260812 00:07:30.437414 27331 leader_election.cc:302] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bd58cede2ec45879b4ace39301a74a0; no voters:
I20260812 00:07:30.437633 27331 leader_election.cc:288] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:07:30.437656 27337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:30.437778 27337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1bd58cede2ec45879b4ace39301a74a0, State: Running, Role: LEADER
I20260812 00:07:30.437873 27337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd58cede2ec45879b4ace39301a74a0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40451 } }
I20260812 00:07:30.437923 27331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:30.438206 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bd58cede2ec45879b4ace39301a74a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd58cede2ec45879b4ace39301a74a0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40451 } } }
I20260812 00:07:30.438282 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:30.438434 27337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bd58cede2ec45879b4ace39301a74a0. Latest consensus state: current_term: 1 leader_uuid: "1bd58cede2ec45879b4ace39301a74a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bd58cede2ec45879b4ace39301a74a0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40451 } } }
I20260812 00:07:30.438510 27337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:07:30.438938 27350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:07:30.439496 27350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:07:30.441529 27350 catalog_manager.cc:1347] Generated new cluster ID: 753fd43cff1e481c80b7f87cf9a485f5
I20260812 00:07:30.441591 27350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:07:30.442780 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.431053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53730 (local address 127.7.85.254:40451)
0812 00:07:30.431259 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:30.431267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:30.431316 (+ 49us) server_negotiation.cc:407] Connection header received
0812 00:07:30.431722 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:30.431728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:30.431799 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:30.431947 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:30.432617 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.434018 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.434842 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.435496 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.435636 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.438879 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:30.438884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:30.438884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:30.438897 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:30.441020 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.441483 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.441486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.441486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.441613 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.441871 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.441872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.441872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.442125 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:30.442300 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:30.442408 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 00:07:30.442555 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260812 00:07:30.454432 27350 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:07:30.455183 27350 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:07:30.458841 27350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0: Generated new TSK 0
I20260812 00:07:30.459050 27350 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:07:30.497469 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40451
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:30.584741 27380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:30.584980 27380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:30.585021 27380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:30.586223 27380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:30.586292 27380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:30.586338 27380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:07:30.587814 27380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:30.588083 27380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:30.588321 27380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:30.593822 27387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.593958 27380 server_base.cc:1034] running on GCE node
W20260812 00:07:30.593814 27386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:30.593796 27389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.594306 27380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:30.594573 27380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:30.595747 27380 hybrid_clock.cc:648] HybridClock initialized: now 1786493250595723 us; error 36 us; skew 500 ppm
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:30.598590 27380 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:07:30.599038 27380 webserver.cc:460] Webserver started at http://127.7.85.193:36799/ using document root <none> and password file <none>
I20260812 00:07:30.599272 27380 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:30.599325 27380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:30.599489 27380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:30.600551 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "4f55addce8f1dd0bc6fe0d357a3e6455"
server_key_iv: "a9b37b73929d83fd86bde12c49f1afc1"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.600911 27380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "4f55addce8f1dd0bc6fe0d357a3e6455"
server_key_iv: "a9b37b73929d83fd86bde12c49f1afc1"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.602347 27380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:07:30.603148 27399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.603402 27380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:07:30.603498 27380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "4f55addce8f1dd0bc6fe0d357a3e6455"
server_key_iv: "a9b37b73929d83fd86bde12c49f1afc1"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.603575 27380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:30.615212 27380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:30.617429 27380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:30.617604 27380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:30.617857 27380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:30.618192 27380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:30.618235 27380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.618258 27380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:30.618309 27380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.624243 27380 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:34471
I20260812 00:07:30.624303 27585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:34471 every 8 connection(s)
I20260812 00:07:30.624603 27380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:30.633962 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 27380
I20260812 00:07:30.634087 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:07:30.634241 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.625320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:47033 (local address 127.7.85.254:40451)
0812 00:07:30.625386 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:30.625388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:30.625677 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:07:30.626137 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:30.626139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:30.626150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:30.626224 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:30.627252 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.627885 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.628939 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.629152 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.629270 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.631442 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:30.631443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:30.631444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:30.631452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:30.632692 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.633111 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.633112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.633113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.633224 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.633480 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.633481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.633481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.633598 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:30.633668 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:30.634055 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 00:07:30.634163 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 00:07:30.634320 7511 external_mini_cluster.cc:1428] Setting key 657f87f6c2dbf721ecd4271f50144e7f
I20260812 00:07:30.634430 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.625308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40451 (local address 127.7.85.193:47033)
0812 00:07:30.625475 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:30.625519 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:30.626013 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:30.626323 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:30.626327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:30.626925 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:30.627116 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:30.627121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.627996 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.627997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:30.628845 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:30.628848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.629330 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.629331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:30.629449 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.629923 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:30.629931 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:30.631341 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:30.632780 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:30.632789 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:30.632792 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:30.633031 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:30.633291 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:30.633292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:30.633292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:30.633403 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:30.633671 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:30.633679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:30.633870 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 00:07:30.634000 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 00:07:30.634188 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260812 00:07:30.634912 27586 heartbeater.cc:347] Connected to a master server at 127.7.85.254:40451
I20260812 00:07:30.635023 27586 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:30.635229 27586 heartbeater.cc:511] Master 127.7.85.254:40451 requested a full tablet report, sending...
I20260812 00:07:30.635795 27273 ts_manager.cc:194] Registered new tserver with Master: 2f512cfcdd564b4fa3e0f87ce4be68d0 (127.7.85.193:34471)
I20260812 00:07:30.636416 27273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:47033
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:07:30.665786 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40451
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:30.752254 27591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:30.752482 27591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:30.752508 27591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:30.753638 27591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:30.753677 27591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:30.753705 27591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:07:30.755137 27591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:30.755386 27591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:30.755575 27591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:30.760967 27604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.761124 27591 server_base.cc:1034] running on GCE node
W20260812 00:07:30.761255 27608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:30.760987 27605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.761515 27591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:30.761806 27591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:30.762977 27591 hybrid_clock.cc:648] HybridClock initialized: now 1786493250762966 us; error 31 us; skew 500 ppm
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:30.765679 27591 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:07:30.766077 27591 webserver.cc:460] Webserver started at http://127.7.85.194:34393/ using document root <none> and password file <none>
I20260812 00:07:30.766265 27591 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:30.766312 27591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:30.766409 27591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:30.767427 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "d35853d0777c42ee8079e119e6ccfaed"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "df079cf0db3015e5d5f5f513f14c39ed"
server_key_iv: "c86e9181f1b4a47d2a2f88b45e407bde"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.767724 27591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "d35853d0777c42ee8079e119e6ccfaed"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "df079cf0db3015e5d5f5f513f14c39ed"
server_key_iv: "c86e9181f1b4a47d2a2f88b45e407bde"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.768996 27591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:07:30.769791 27621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.769955 27591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.770040 27591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "d35853d0777c42ee8079e119e6ccfaed"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "df079cf0db3015e5d5f5f513f14c39ed"
server_key_iv: "c86e9181f1b4a47d2a2f88b45e407bde"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.770107 27591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:30.782619 27591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:30.785033 27591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:30.785241 27591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:30.785513 27591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:30.785856 27591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:30.785893 27591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.785945 27591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:30.785974 27591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.792420 27591 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:33211
I20260812 00:07:30.792491 27823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:33211 every 8 connection(s)
I20260812 00:07:30.792847 27591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:30.802515 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 27591
I20260812 00:07:30.802634 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:07:30.802893 7511 external_mini_cluster.cc:1428] Setting key f52db6daf11a3fcfffdfdf39db6613c7
I20260812 00:07:30.802930 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.793679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:42203 (local address 127.7.85.254:40451)
0812 00:07:30.793747 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:30.793749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:30.793975 (+ 226us) server_negotiation.cc:407] Connection header received
0812 00:07:30.794509 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:30.794510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:30.794524 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:30.794589 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:30.795556 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.796052 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.797048 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.797299 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.797387 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.799565 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:30.799567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:30.799567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:30.799575 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:30.801274 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.801706 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.801707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.801708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.801772 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.802030 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.802031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.802031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.802154 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:30.802231 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:30.802640 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 00:07:30.802757 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 00:07:30.803018 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.793664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40451 (local address 127.7.85.194:42203)
0812 00:07:30.793839 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:30.793887 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:30.794402 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:30.794658 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:30.794662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:30.795262 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:30.795457 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:30.795461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.796146 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.796147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:30.796945 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:30.796948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.797400 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.797402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:30.797525 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.798072 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:30.798081 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:30.799473 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:30.801359 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:30.801367 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:30.801370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:30.801612 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:30.801829 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:30.801830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:30.801831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:30.801955 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:30.802222 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:30.802230 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:30.802429 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 00:07:30.802580 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 00:07:30.802760 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260812 00:07:30.804234 27825 heartbeater.cc:347] Connected to a master server at 127.7.85.254:40451
I20260812 00:07:30.804456 27825 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:30.804694 27825 heartbeater.cc:511] Master 127.7.85.254:40451 requested a full tablet report, sending...
I20260812 00:07:30.805126 27273 ts_manager.cc:194] Registered new tserver with Master: d35853d0777c42ee8079e119e6ccfaed (127.7.85.194:33211)
I20260812 00:07:30.805683 27273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:42203
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:07:30.837402 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40451
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:07:30.926245 27833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:07:30.926499 27833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:07:30.926539 27833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:07:30.927685 27833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:07:30.927752 27833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:07:30.927798 27833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:07:30.929351 27833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:40451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:07:30.929623 27833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:07:30.929873 27833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:07:30.935266 27841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:30.935261 27843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:07:30.935259 27845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:07:30.935562 27833 server_base.cc:1034] running on GCE node
I20260812 00:07:30.935822 27833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:07:30.936085 27833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:07:30.937287 27833 hybrid_clock.cc:648] HybridClock initialized: now 1786493250937255 us; error 47 us; skew 500 ppm
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:07:30.940240 27833 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:07:30.940694 27833 webserver.cc:460] Webserver started at http://127.7.85.195:42415/ using document root <none> and password file <none>
I20260812 00:07:30.940907 27833 fs_manager.cc:359] Metadata directory not provided
I20260812 00:07:30.940950 27833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:07:30.941046 27833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:07:30.942116 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "ab27a88e20154487871bcf41f9800d04"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "3d217933e60592e473371fec1e149996"
server_key_iv: "126c758520abd0c60dc7716202e652ff"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.942437 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "ab27a88e20154487871bcf41f9800d04"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "3d217933e60592e473371fec1e149996"
server_key_iv: "126c758520abd0c60dc7716202e652ff"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.943730 27833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:07:30.944510 27861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.944725 27833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:07:30.944837 27833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "ab27a88e20154487871bcf41f9800d04"
format_stamp: "Formatted at 2026-08-12 00:07:30 on dist-test-slave-z8x9"
server_key: "3d217933e60592e473371fec1e149996"
server_key_iv: "126c758520abd0c60dc7716202e652ff"
server_key_version: "encryptionkey@0"
I20260812 00:07:30.944916 27833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:07:30.961655 27833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:07:30.964172 27833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:07:30.964356 27833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:07:30.964623 27833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:07:30.964970 27833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:07:30.965026 27833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.965071 27833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:07:30.965102 27833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:07:30.971318 27833 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:46457
I20260812 00:07:30.971357 28064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:46457 every 8 connection(s)
I20260812 00:07:30.971709 27833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:07:30.973984 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 27833
I20260812 00:07:30.974117 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:07:30.974323 7511 external_mini_cluster.cc:1428] Setting key 170b5319cc2fb8ce591d35c6343eb3bc
Aug 12 00:07:30 dist-test-slave-z8x9 krb5kdc[27179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:07:30.982787 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.972495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:34331 (local address 127.7.85.254:40451)
0812 00:07:30.972553 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:30.972556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:30.972833 (+ 277us) server_negotiation.cc:407] Connection header received
0812 00:07:30.973398 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:30.973399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:30.973410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:30.973460 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:30.974827 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.975714 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.976757 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.976996 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.977072 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.979836 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:30.979838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:30.979838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:30.979846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:30.981324 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.981749 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.981750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.981751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.981816 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.982086 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.982087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.982088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.982230 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:30.982327 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:30.982632 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 00:07:30.982718 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:07:30.982929 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.972490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40451 (local address 127.7.85.195:34331)
0812 00:07:30.972658 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:30.972736 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:30.973310 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:30.973520 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:30.973523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:30.974377 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:07:30.974638 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:30.974645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.975835 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.975836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:30.976647 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:30.976650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.977116 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.977117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:30.977342 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.977960 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:07:30.977970 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:07:30.979731 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:07:30.981403 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:30.981410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:30.981412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:30.981652 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:30.981884 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:07:30.981885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:07:30.981885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:07:30.981997 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:07:30.982323 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:07:30.982329 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:07:30.982472 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 00:07:30.982566 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 00:07:30.982704 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260812 00:07:30.983522 28065 heartbeater.cc:347] Connected to a master server at 127.7.85.254:40451
I20260812 00:07:30.983644 28065 heartbeater.cc:464] Registering TS with master...
I20260812 00:07:30.983840 28065 heartbeater.cc:511] Master 127.7.85.254:40451 requested a full tablet report, sending...
I20260812 00:07:30.984314 27273 ts_manager.cc:194] Registered new tserver with Master: ab27a88e20154487871bcf41f9800d04 (127.7.85.195:46457)
I20260812 00:07:30.984652 27273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:34331
I20260812 00:07:30.986629 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:07:30.994396 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:30.987980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.7.85.254:40451)
0812 00:07:30.988039 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:30.988041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:30.988179 (+ 138us) server_negotiation.cc:407] Connection header received
0812 00:07:30.988254 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:30.988255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:30.988265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:30.988318 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:07:30.988879 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.989464 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.990210 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:30.990433 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:30.990507 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:30.991777 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:07:30.991778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:07:30.991778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:07:30.991785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:07:30.993297 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.993665 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.993666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.993666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.993743 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:07:30.993925 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:07:30.993926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:07:30.993926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:07:30.994081 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:07:30.994164 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:30.994253 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 00:07:30.994318 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 00:07:30.996131 27273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:07:30.997982 27273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:07:31.005270 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.000845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33211 (local address 127.0.0.1:38482)
0812 00:07:31.001251 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.001259 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.001315 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.001970 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.001971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.001974 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.002028 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.002029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.003193 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.003194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.003926 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.003928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.004906 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.004907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.004995 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.005012 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.005088 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.005153 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":70,"threads_started":1}
I20260812 00:07:31.005561 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.000630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:34471 (local address 127.0.0.1:44934)
0812 00:07:31.001058 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.001068 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.001117 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.001619 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.001620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.001623 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.001658 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.001659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.002973 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.002975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.004155 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.004158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.005204 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.005205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.005273 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.005277 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.005421 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.005486 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":105,"threads_started":1}
I20260812 00:07:31.005621 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.001098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.7.85.194:33211)
0812 00:07:31.001802 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.001806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.001815 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:07:31.001858 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.001860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.001875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.001956 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.002105 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.003102 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.004019 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.004736 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.004906 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.004984 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.005487 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.005535 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":128,"threads_started":1}
I20260812 00:07:31.005882 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.000742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39536 (local address 127.7.85.195:46457)
0812 00:07:31.000953 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.000957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.001098 (+ 141us) server_negotiation.cc:407] Connection header received
0812 00:07:31.001222 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.001225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.001244 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.001338 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.001516 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.002532 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.004141 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.004985 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.005068 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.005130 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.005599 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.005772 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260812 00:07:31.006036 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.000627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:46457 (local address 127.0.0.1:39536)
0812 00:07:31.000972 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.001017 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.001104 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.001350 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.001353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.001355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.001444 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.001446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.002635 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.002636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.004031 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.004034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.005070 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.005072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.005150 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.005155 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.005270 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.005321 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":67,"threads_started":1}
I20260812 00:07:31.006359 27716 tablet_service.cc:1467] Processing CreateTablet for tablet 243c79149efa49a6b44e3fc29adecd54 (DEFAULT_TABLE table=test-table [id=eeffd52993074a509ca62d9dd70d4313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:31.006718 27941 tablet_service.cc:1467] Processing CreateTablet for tablet 243c79149efa49a6b44e3fc29adecd54 (DEFAULT_TABLE table=test-table [id=eeffd52993074a509ca62d9dd70d4313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:31.006868 27716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 243c79149efa49a6b44e3fc29adecd54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:31.007102 27941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 243c79149efa49a6b44e3fc29adecd54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:31.009725 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.000740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.7.85.193:34471)
0812 00:07:31.000909 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.000915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.001377 (+ 462us) server_negotiation.cc:407] Connection header received
0812 00:07:31.001426 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.001429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.001445 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.001519 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.001725 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.002850 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.004241 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.005068 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.005202 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.005266 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.005613 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.005815 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260812 00:07:31.009968 28094 tablet_bootstrap.cc:492] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Bootstrap starting.
I20260812 00:07:31.010408 27484 tablet_service.cc:1467] Processing CreateTablet for tablet 243c79149efa49a6b44e3fc29adecd54 (DEFAULT_TABLE table=test-table [id=eeffd52993074a509ca62d9dd70d4313]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:07:31.010694 28094 tablet_bootstrap.cc:654] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:31.010744 27484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 243c79149efa49a6b44e3fc29adecd54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:07:31.011296 28094 log.cc:824] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:31.012144 28094 tablet_bootstrap.cc:492] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: No bootstrap required, opened a new log
I20260812 00:07:31.012226 28094 ts_tablet_manager.cc:1397] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:07:31.013403 28096 tablet_bootstrap.cc:492] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Bootstrap starting.
I20260812 00:07:31.014006 28096 tablet_bootstrap.cc:654] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:31.014242 28098 tablet_bootstrap.cc:492] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: Bootstrap starting.
I20260812 00:07:31.014492 28094 raft_consensus.cc:348] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.014652 28094 raft_consensus.cc:374] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:31.014698 28094 raft_consensus.cc:729] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35853d0777c42ee8079e119e6ccfaed, State: Initialized, Role: FOLLOWER
I20260812 00:07:31.014701 28096 log.cc:824] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:31.014817 28094 consensus_queue.cc:260] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.014995 28098 tablet_bootstrap.cc:654] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: Neither blocks nor log segments found. Creating new log.
I20260812 00:07:31.015144 28094 ts_tablet_manager.cc:1428] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:07:31.015164 27825 heartbeater.cc:503] Master 127.7.85.254:40451 was elected leader, sending a full tablet report...
I20260812 00:07:31.015604 28098 log.cc:824] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:07:31.015697 28096 tablet_bootstrap.cc:492] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: No bootstrap required, opened a new log
I20260812 00:07:31.015760 28096 ts_tablet_manager.cc:1397] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:31.016342 28098 tablet_bootstrap.cc:492] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: No bootstrap required, opened a new log
I20260812 00:07:31.016422 28098 ts_tablet_manager.cc:1397] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:07:31.018007 28096 raft_consensus.cc:348] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.018188 28096 raft_consensus.cc:374] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:31.018227 28096 raft_consensus.cc:729] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab27a88e20154487871bcf41f9800d04, State: Initialized, Role: FOLLOWER
I20260812 00:07:31.018355 28096 consensus_queue.cc:260] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.018661 28096 ts_tablet_manager.cc:1428] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:07:31.018693 28065 heartbeater.cc:503] Master 127.7.85.254:40451 was elected leader, sending a full tablet report...
I20260812 00:07:31.018625 28098 raft_consensus.cc:348] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.018806 28098 raft_consensus.cc:374] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:07:31.018874 28098 raft_consensus.cc:729] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f512cfcdd564b4fa3e0f87ce4be68d0, State: Initialized, Role: FOLLOWER
I20260812 00:07:31.018981 28098 consensus_queue.cc:260] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.019259 28098 ts_tablet_manager.cc:1428] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:07:31.019297 27586 heartbeater.cc:503] Master 127.7.85.254:40451 was elected leader, sending a full tablet report...
W20260812 00:07:31.043522 27827 tablet.cc:2369] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:07:31.125349 27587 tablet.cc:2369] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:07:31.222327 28067 tablet.cc:2369] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:07:31.386185 28102 raft_consensus.cc:482] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:31.386346 28102 raft_consensus.cc:504] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.386832 28102 leader_election.cc:288] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f512cfcdd564b4fa3e0f87ce4be68d0 (127.7.85.193:34471), ab27a88e20154487871bcf41f9800d04 (127.7.85.195:46457)
I20260812 00:07:31.390175 28105 raft_consensus.cc:482] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:07:31.390311 28105 raft_consensus.cc:504] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.390779 28105 leader_election.cc:288] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f512cfcdd564b4fa3e0f87ce4be68d0 (127.7.85.193:34471), d35853d0777c42ee8079e119e6ccfaed (127.7.85.194:33211)
I20260812 00:07:31.391535 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.387234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:60777 (local address 127.7.85.195:46457)
0812 00:07:31.387295 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.387297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.387311 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:07:31.387432 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.387433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.387446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.387535 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.387764 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.388793 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.390431 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.391080 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.391125 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.391137 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.391399 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.391448 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:07:31.391664 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.387314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:37829 (local address 127.7.85.193:34471)
0812 00:07:31.387361 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.387363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.387437 (+ 74us) server_negotiation.cc:407] Connection header received
0812 00:07:31.387485 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.387486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.387498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.387560 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.387770 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.388791 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.390592 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.391173 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.391260 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.391269 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.391534 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.391581 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:07:31.391805 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.387083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:46457 (local address 127.7.85.194:60777)
0812 00:07:31.387248 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.387260 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.387339 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.387576 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.387577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.387581 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.387695 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.387698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.388933 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.388934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.390317 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.390321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.391207 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.391208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.391295 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.391299 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.391657 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.391719 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":29,"thread_start_us":92,"threads_started":1}
I20260812 00:07:31.392086 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.387083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:34471 (local address 127.7.85.194:37829)
0812 00:07:31.387338 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.387348 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.387425 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.387577 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.387577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.387579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.387695 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.387698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.388932 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.388934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.390440 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.390442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.391347 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.391348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.391438 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.391444 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.391955 (+ 511us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.391998 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":90,"thread_start_us":98,"threads_started":1}
I20260812 00:07:31.392123 27980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243c79149efa49a6b44e3fc29adecd54" candidate_uuid: "d35853d0777c42ee8079e119e6ccfaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab27a88e20154487871bcf41f9800d04" is_pre_election: true
I20260812 00:07:31.392270 27980 raft_consensus.cc:2454] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35853d0777c42ee8079e119e6ccfaed in term 0.
I20260812 00:07:31.392442 27516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243c79149efa49a6b44e3fc29adecd54" candidate_uuid: "d35853d0777c42ee8079e119e6ccfaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" is_pre_election: true
I20260812 00:07:31.392494 27622 leader_election.cc:302] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab27a88e20154487871bcf41f9800d04, d35853d0777c42ee8079e119e6ccfaed; no voters:
I20260812 00:07:31.392613 27516 raft_consensus.cc:2454] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35853d0777c42ee8079e119e6ccfaed in term 0.
I20260812 00:07:31.392647 28102 raft_consensus.cc:2781] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:07:31.392701 28102 raft_consensus.cc:482] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:07:31.392733 28102 raft_consensus.cc:3037] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:31.393590 28102 raft_consensus.cc:504] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.393751 28102 leader_election.cc:288] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [CANDIDATE]: Term 1 election: Requested vote from peers 2f512cfcdd564b4fa3e0f87ce4be68d0 (127.7.85.193:34471), ab27a88e20154487871bcf41f9800d04 (127.7.85.195:46457)
I20260812 00:07:31.394003 27980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243c79149efa49a6b44e3fc29adecd54" candidate_uuid: "d35853d0777c42ee8079e119e6ccfaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab27a88e20154487871bcf41f9800d04"
I20260812 00:07:31.394096 27980 raft_consensus.cc:3037] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:31.394411 27516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243c79149efa49a6b44e3fc29adecd54" candidate_uuid: "d35853d0777c42ee8079e119e6ccfaed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0"
I20260812 00:07:31.394500 27516 raft_consensus.cc:3037] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:07:31.394795 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.390988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:33211 (local address 127.7.85.195:36527)
0812 00:07:31.391389 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.391399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.391479 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.391651 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.391651 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.391654 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.391739 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.391741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.392595 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.392596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.393587 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.393591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.394370 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.394372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.394450 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.394455 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.394610 (+ 155us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.394696 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":41,"thread_start_us":98,"threads_started":1}
I20260812 00:07:31.395025 27980 raft_consensus.cc:2454] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35853d0777c42ee8079e119e6ccfaed in term 1.
I20260812 00:07:31.395285 27622 leader_election.cc:302] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab27a88e20154487871bcf41f9800d04, d35853d0777c42ee8079e119e6ccfaed; no voters:
I20260812 00:07:31.395545 27516 raft_consensus.cc:2454] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35853d0777c42ee8079e119e6ccfaed in term 1.
I20260812 00:07:31.395807 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.391237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:36527 (local address 127.7.85.194:33211)
0812 00:07:31.391279 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.391282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.391453 (+ 171us) server_negotiation.cc:407] Connection header received
0812 00:07:31.391556 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.391556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.391566 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.391594 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.391829 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.392493 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.393691 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.394275 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.394358 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.394366 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.395695 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.395744 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:07:31.395844 28102 raft_consensus.cc:2781] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:07:31.396023 28102 raft_consensus.cc:686] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 1 LEADER]: Becoming Leader. State: Replica: d35853d0777c42ee8079e119e6ccfaed, State: Running, Role: LEADER
I20260812 00:07:31.396148 28102 consensus_queue.cc:237] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } }
I20260812 00:07:31.396179 27747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243c79149efa49a6b44e3fc29adecd54" candidate_uuid: "ab27a88e20154487871bcf41f9800d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35853d0777c42ee8079e119e6ccfaed" is_pre_election: true
I20260812 00:07:31.397264 27273 catalog_manager.cc:5495] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed reported cstate change: term changed from 0 to 1, leader changed from <none> to d35853d0777c42ee8079e119e6ccfaed (127.7.85.194). New cstate: current_term: 1 leader_uuid: "d35853d0777c42ee8079e119e6ccfaed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35853d0777c42ee8079e119e6ccfaed" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 33211 } health_report { overall_health: HEALTHY } } }
I20260812 00:07:31.397629 28068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.390918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:34471 (local address 127.7.85.195:60343)
0812 00:07:31.391024 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.391035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:07:31.391073 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:07:31.391523 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:07:31.391524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:07:31.391526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:07:31.391590 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.391591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.394568 (+ 2977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.394569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.395735 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:07:31.395750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.397286 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.397288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:07:31.397385 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.397389 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:07:31.397479 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:07:31.397525 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 00:07:31.397830 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.391241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:60343 (local address 127.7.85.193:34471)
0812 00:07:31.391391 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.391393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.391400 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:07:31.391422 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.391422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.391432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.391659 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:07:31.391693 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.394478 (+ 2785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.396003 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.396965 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.397048 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.397060 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.397500 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.397551 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260812 00:07:31.398022 27516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "243c79149efa49a6b44e3fc29adecd54" candidate_uuid: "ab27a88e20154487871bcf41f9800d04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" is_pre_election: true
I20260812 00:07:31.398128 27516 raft_consensus.cc:2379] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ab27a88e20154487871bcf41f9800d04 in current term 1: Already voted for candidate d35853d0777c42ee8079e119e6ccfaed in this term.
I20260812 00:07:31.398442 27865 leader_election.cc:302] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab27a88e20154487871bcf41f9800d04; no voters: 2f512cfcdd564b4fa3e0f87ce4be68d0, d35853d0777c42ee8079e119e6ccfaed
I20260812 00:07:31.398586 28105 raft_consensus.cc:2726] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 00:07:31.447264 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.444377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.7.85.194:33211)
0812 00:07:31.444451 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 00:07:31.444453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:07:31.444461 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:07:31.444501 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:07:31.444502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:07:31.444522 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:07:31.444594 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:07:31.444758 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.445573 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.446393 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:07:31.446618 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:07:31.446700 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:07:31.446910 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:07:31.447031 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:07:31.447107 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:07:31.447170 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 00:07:31.447193 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:07:31.449455 27516 raft_consensus.cc:1261] T 243c79149efa49a6b44e3fc29adecd54 P 2f512cfcdd564b4fa3e0f87ce4be68d0 [term 1 FOLLOWER]: Refusing update from remote peer d35853d0777c42ee8079e119e6ccfaed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:31.449496 27980 raft_consensus.cc:1261] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 1 FOLLOWER]: Refusing update from remote peer d35853d0777c42ee8079e119e6ccfaed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:07:31.449793 28123 consensus_queue.cc:1035] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f512cfcdd564b4fa3e0f87ce4be68d0" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 34471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:31.449930 28102 consensus_queue.cc:1035] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab27a88e20154487871bcf41f9800d04" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:07:31.452684 28127 mvcc.cc:205] Tried to move back new op lower bound from 7317476357933645824 to 7317476357719232512. Current Snapshot: MvccSnapshot[applied={T|T < 7317476357933645824}]
I20260812 00:07:31.455215 27273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:31.455329 27273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:07:31.456260 27273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bd58cede2ec45879b4ace39301a74a0: Sending DeleteTablet for 3 replicas of tablet 243c79149efa49a6b44e3fc29adecd54
I20260812 00:07:31.456662 27941 tablet_service.cc:1514] Processing DeleteTablet for tablet 243c79149efa49a6b44e3fc29adecd54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:31 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39536
I20260812 00:07:31.456739 27716 tablet_service.cc:1514] Processing DeleteTablet for tablet 243c79149efa49a6b44e3fc29adecd54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:31 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:38482
I20260812 00:07:31.456738 27484 tablet_service.cc:1514] Processing DeleteTablet for tablet 243c79149efa49a6b44e3fc29adecd54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:07:31 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:44934
I20260812 00:07:31.456947 28145 tablet_replica.cc:330] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: stopping tablet replica
I20260812 00:07:31.457043 28145 raft_consensus.cc:2229] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:07:31.457053 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 27380
I20260812 00:07:31.457222 28145 raft_consensus.cc:2258] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:31.458009 28144 tablet_replica.cc:330] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: stopping tablet replica
I20260812 00:07:31.458180 28144 raft_consensus.cc:2229] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:07:31.458190 28145 ts_tablet_manager.cc:1905] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:31.458784 28144 raft_consensus.cc:2258] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:07:31.460975 28145 ts_tablet_manager.cc:1918] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:31.461059 28145 log.cc:1196] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/243c79149efa49a6b44e3fc29adecd54
I20260812 00:07:31.461367 28145 ts_tablet_manager.cc:1939] T 243c79149efa49a6b44e3fc29adecd54 P d35853d0777c42ee8079e119e6ccfaed: Deleting consensus metadata
I20260812 00:07:31.461562 28144 ts_tablet_manager.cc:1905] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:07:31.461830 27241 catalog_manager.cc:4841] TS d35853d0777c42ee8079e119e6ccfaed (127.7.85.194:33211): tablet 243c79149efa49a6b44e3fc29adecd54 (table test-table [id=eeffd52993074a509ca62d9dd70d4313]) successfully deleted
I20260812 00:07:31.463294 28144 ts_tablet_manager.cc:1918] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:07:31.463480 28144 log.cc:1196] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/243c79149efa49a6b44e3fc29adecd54
I20260812 00:07:31.463835 28144 ts_tablet_manager.cc:1939] T 243c79149efa49a6b44e3fc29adecd54 P ab27a88e20154487871bcf41f9800d04: Deleting consensus metadata
I20260812 00:07:31.464505 27241 catalog_manager.cc:4841] TS ab27a88e20154487871bcf41f9800d04 (127.7.85.195:46457): tablet 243c79149efa49a6b44e3fc29adecd54 (table test-table [id=eeffd52993074a509ca62d9dd70d4313]) successfully deleted
W20260812 00:07:31.475024 27246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:34471 (error 108)
I20260812 00:07:31.475732 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:07:31.475578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:34471 (local address 127.0.0.1:44948)
0812 00:07:31.475641 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 00:07:31.475689 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 00:07:31.475862 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 27591
W20260812 00:07:31.482235 27246 catalog_manager.cc:4584] TS 2f512cfcdd564b4fa3e0f87ce4be68d0 (127.7.85.193:34471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 243c79149efa49a6b44e3fc29adecd54: Network error: Client connection negotiation failed: client connection to 127.7.85.193:34471: connect: Connection refused (error 111)
I20260812 00:07:31.487273 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 27833
I20260812 00:07:31.507987 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 27201
2026-08-12T00:07:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2688 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:07:31 dist-test-slave-z8x9 krb5kdc[28153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:07:31 dist-test-slave-z8x9 krb5kdc[28153](info): set up 2 sockets
Aug 12 00:07:31 dist-test-slave-z8x9 krb5kdc[28153](info): commencing operation
krb5kdc: starting...
W20260812 00:07:32.915202 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.376s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:07:33 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:07:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:07:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/rangeradmin_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/rangeradmin_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/rangerlookup_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/rangerlookup_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
I20260812 00:07:33.138123 7511 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskaxXthI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/postgres -l logfile start
2026-08-12 00:07:36.483 UTC [28207] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 00:07:36.483 UTC [28207] LOG: listening on IPv4 address "127.7.85.212", port 50119
2026-08-12 00:07:36.489 UTC [28207] LOG: listening on Unix socket "/tmp/.s.PGSQL.50119"
2026-08-12 00:07:36.516 UTC [28215] LOG: database system was shut down at 2026-08-12 00:07:34 UTC
2026-08-12 00:07:36.520 UTC [28207] LOG: database system is ready to accept connections
I20260812 00:07:37.801326 7511 mini_postgres.cc:96] Postgres bound to 50119
2026-08-12 00:07:37.810 UTC [28226] FATAL: database "slave" does not exist
127.7.85.212:50119 - accepting connections
I20260812 00:07:37.811357 7511 mini_ranger.cc:161] Starting Ranger...
I20260812 00:07:37.831132 7511 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 00:07:38.592980 7511 mini_ranger.cc:87] Created ranger Postgres database
I20260812 00:07:38.593073 7511 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-admin
2026-08-12 00:07:38,919 [I] DB FLAVOR :POSTGRES
2026-08-12 00:07:38,920 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:07:38,920 [I] Checking connection..
2026-08-12 00:07:38,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 00:07:39,223 [I] Checking connection passed.
2026-08-12 00:07:39,224 [I] --------- Verifying version history table ---------
2026-08-12 00:07:39,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:07:39,500 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:07:39,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:07:39,789 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:07:39,790 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 00:07:39,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 00:07:40.064 UTC [28339] WARNING: there is no transaction in progress
2026-08-12 00:07:40,077 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 00:07:40,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:07:40,376 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 00:07:40,377 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:07:40,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 00:07:40,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 00:07:40,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z8x9.c.gcp-upstream.internal','N') ;"
2026-08-12 00:07:41,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:07:41,543 [I] Table x_portal_user does not exist in database ranger
2026-08-12 00:07:41,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:07:41,826 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 00:07:41,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:07:42,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:07:42,384 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 00:07:42,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 00:07:43.600 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.620 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.641 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.663 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.683 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.710 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.792 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.801 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.818 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.830 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.842 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.863 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.875 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.883 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.893 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:43.902 UTC [28538] WARNING: there is no transaction in progress
2026-08-12 00:07:44,885 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 00:07:44,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 00:07:45,152 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 00:07:45,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:07:45,431 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 00:07:45,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:07:45,710 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 00:07:45,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:07:45,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:07:46,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z8x9.c.gcp-upstream.internal';"
2026-08-12 00:07:46,564 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 00:07:46,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 00:07:46,858 [I] DB_PATCHES have already been applied
I20260812 00:07:46.865396 7511 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-admin:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskaxXthI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 00:07:46.865500 7511 mini_ranger.cc:193] Using host: 127.7.85.212
I20260812 00:07:46.872020 7511 mini_ranger.cc:227] Ranger admin URL: http://127.7.85.212:46377
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 12:07:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 00:07:48.339445 7511 mini_ranger.cc:160] Time spent starting Ranger: real 10.528s user 0.004s sys 0.010s
Aug 12, 2026 12:08:10 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 12:08:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 12:08:10 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 12:08:10 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 12:08:11 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 00:08:12.014824 7511 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/rangerkms_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/rangerkms_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.85.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.85.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 00:08:12 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493292, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 00:08:12.135886 7511 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 00:08:12.149627 7511 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 00:08:13.329741 7511 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 00:08:13.329852 7511 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms
I20260812 00:08:13.329873 7511 mini_ranger_kms.cc:227] Using postgres at 127.7.85.212:50119
2026-08-12 00:08:14,085 [I] DB FLAVOR :POSTGRES
2026-08-12 00:08:14,086 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:08:14,086 [I] Checking connection
2026-08-12 00:08:14,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 00:08:14,454 [I] connection success
2026-08-12 00:08:14,455 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 00:08:14,455 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 00:08:14,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 00:08:14,745 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 00:08:14,745 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:08:14,745 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 00:08:14,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:50119/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 00:08:15,047 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 00:08:15.609416 7511 mini_ranger_kms.cc:310] Created kms service
I20260812 00:08:15.823164 7511 mini_ranger_kms.cc:326] Created kudu user
I20260812 00:08:15.870687 7511 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 00:08:15 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493292, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:08:16.303164 7511 mini_ranger_kms.cc:384] Added ranger policy
I20260812 00:08:16.303310 7511 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms:/tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 00:08:16.303335 7511 mini_ranger_kms.cc:240] Using host: 127.7.85.212
I20260812 00:08:16.308496 7511 mini_ranger_kms.cc:276] Ranger KMS PID: 28877
I20260812 00:08:16.308631 7511 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.85.212:52937
Aug 12, 2026 12:08:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:08:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:08:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 12:08:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 12:08:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 12:08:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 00:08:19 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786493299, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.85.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:08:19.381902 7511 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.246s user 0.000s sys 0.718s
I20260812 00:08:19.382028 7511 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 00:08:19.382119 7511 mini_ranger_kms.cc:401] 127.7.85.212:52937/kms/v1/keys
Aug 12 00:08:19 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493299, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.85.212@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
Aug 12 00:08:20 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493299, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.85.212@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
Aug 12 00:08:23 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:08:23.382484 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:43909
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:43909
--ranger_config_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:52937/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:08:23.515817 28959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:08:23.516067 28959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:08:23.516093 28959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:08:23.517382 28959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:08:23.517423 28959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:08:23.517437 28959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:08:23.517450 28959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:08:23.517462 28959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:08:23.519227 28959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.85.212:52937/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:43909
--ranger_config_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:43909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:08:23.519448 28959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:08:23.519639 28959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:08:23.526584 28966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:08:23.526649 28959 server_base.cc:1034] running on GCE node
W20260812 00:08:23.526615 28969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:23.526630 28967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:08:23.527307 28959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:08:23.527691 28959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:08:23.528896 28959 hybrid_clock.cc:648] HybridClock initialized: now 1786493303528818 us; error 101 us; skew 500 ppm
Aug 12 00:08:23 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493303, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:08:23.532536 28959 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:08:23.533093 28959 webserver.cc:460] Webserver started at http://127.7.85.254:39393/ using document root <none> and password file <none>
I20260812 00:08:23.533381 28959 fs_manager.cc:359] Metadata directory not provided
I20260812 00:08:23.533425 28959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:08:23.533528 28959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:08:23 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493303, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:08:23.593763 28959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "6392fda2fb404a3e96732417430872eb"
format_stamp: "Formatted at 2026-08-12 00:08:23 on dist-test-slave-z8x9"
server_key: "b1e62ae47a916c403fe231ebab5392f1"
server_key_iv: "5133341a493c56c696a78693562a1391"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:23.594355 28959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "6392fda2fb404a3e96732417430872eb"
format_stamp: "Formatted at 2026-08-12 00:08:23 on dist-test-slave-z8x9"
server_key: "b1e62ae47a916c403fe231ebab5392f1"
server_key_iv: "5133341a493c56c696a78693562a1391"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:23.596437 28959 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:08:23.613641 28988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:23.614001 28959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:08:23.614114 28959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "6392fda2fb404a3e96732417430872eb"
format_stamp: "Formatted at 2026-08-12 00:08:23 on dist-test-slave-z8x9"
server_key: "b1e62ae47a916c403fe231ebab5392f1"
server_key_iv: "5133341a493c56c696a78693562a1391"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:23.614228 28959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:08:23.643407 28959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:08:23.646385 28959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:08:23.646692 28959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:08:23.659842 28959 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:43909
I20260812 00:08:23.660094 29086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:43909 every 8 connection(s)
I20260812 00:08:23.660455 28959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:08:23.661665 29087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:08:23.663343 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 28959
I20260812 00:08:23.663460 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:08:23.663715 7511 external_mini_cluster.cc:1428] Setting key 9bcc00ce50bb466a15c81bc18179b8db
I20260812 00:08:23.666179 29087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb: Bootstrap starting.
I20260812 00:08:23.667479 29087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb: Neither blocks nor log segments found. Creating new log.
I20260812 00:08:23.668454 29087 log.cc:824] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb: Log is configured to *not* fsync() on all Append() calls
I20260812 00:08:23.669947 29087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb: No bootstrap required, opened a new log
Aug 12 00:08:23 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:08:23.673851 29087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392fda2fb404a3e96732417430872eb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43909 } }
I20260812 00:08:23.674041 29087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:08:23.674093 29087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6392fda2fb404a3e96732417430872eb, State: Initialized, Role: FOLLOWER
I20260812 00:08:23.674271 29087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392fda2fb404a3e96732417430872eb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43909 } }
I20260812 00:08:23.674382 29087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:08:23.674446 29087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:08:23.674503 29087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:08:23.675690 29087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392fda2fb404a3e96732417430872eb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43909 } }
I20260812 00:08:23.675930 29087 leader_election.cc:302] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6392fda2fb404a3e96732417430872eb; no voters:
I20260812 00:08:23.676440 29087 leader_election.cc:288] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:08:23.676578 29095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:08:23.676819 29095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [term 1 LEADER]: Becoming Leader. State: Replica: 6392fda2fb404a3e96732417430872eb, State: Running, Role: LEADER
I20260812 00:08:23.676903 29087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:08:23.676924 29095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392fda2fb404a3e96732417430872eb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43909 } }
I20260812 00:08:23.677564 29095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6392fda2fb404a3e96732417430872eb. Latest consensus state: current_term: 1 leader_uuid: "6392fda2fb404a3e96732417430872eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392fda2fb404a3e96732417430872eb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43909 } } }
I20260812 00:08:23.677660 29095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [sys.catalog]: This master's current role is: LEADER
I20260812 00:08:23.678048 29098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6392fda2fb404a3e96732417430872eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6392fda2fb404a3e96732417430872eb" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 43909 } } }
I20260812 00:08:23.678229 29098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb [sys.catalog]: This master's current role is: LEADER
I20260812 00:08:23.678165 29100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:08:23.679211 29100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:08:23.679834 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:23.664332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.7.85.254:43909)
0812 00:08:23.664659 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:23.664669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:23.664712 (+ 43us) server_negotiation.cc:407] Connection header received
0812 00:08:23.665541 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:23.665550 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:23.665603 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:23.665736 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:08:23.666729 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:23.668140 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:23.669297 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:23.670100 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:23.670289 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:23.673767 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:08:23.673773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:08:23.673773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:08:23.673790 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:08:23.676290 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:23.676823 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:23.676828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:23.676828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:23.677010 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:23.677371 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:23.677372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:23.677373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:23.677765 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:08:23.679116 (+ 1351us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:23.679234 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 00:08:23.679445 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":175,"threads_started":1}
I20260812 00:08:23.681732 29087 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 00:08:23.690332 29100 catalog_manager.cc:1347] Generated new cluster ID: 4e1e9bfd69104cbe932682017a549abb
I20260812 00:08:23.690444 29100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:08:23.711068 29100 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:08:23.712934 29100 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:08:23.727421 29100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb: Generated new TSK 0
I20260812 00:08:23.727730 29100 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:08:24.761631 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:52937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43909
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:08:24.902632 29161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:08:24.902870 29161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:08:24.902897 29161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:08:24.904208 29161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:08:24.904263 29161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:08:24.904291 29161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:08:24.906087 29161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.85.212:52937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:08:24.906399 29161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:08:24.906687 29161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:08:24.914458 29169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:24.914520 29168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:24.914876 29172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:08:24.914889 29161 server_base.cc:1034] running on GCE node
I20260812 00:08:24.915166 29161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:08:24.915707 29161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:08:24.917092 29161 hybrid_clock.cc:648] HybridClock initialized: now 1786493304917067 us; error 34 us; skew 500 ppm
Aug 12 00:08:24 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493304, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:08:24.922375 29161 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:08:24.923180 29161 webserver.cc:460] Webserver started at http://127.7.85.193:38651/ using document root <none> and password file <none>
I20260812 00:08:24.923565 29161 fs_manager.cc:359] Metadata directory not provided
I20260812 00:08:24.923637 29161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:08:24.923787 29161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:08:24 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493304, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:08:24.939970 29161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "1d5d5b979cab4e23b266ff10faf06f50"
format_stamp: "Formatted at 2026-08-12 00:08:24 on dist-test-slave-z8x9"
server_key: "749709948b59aabc15dc3fcb072ad7f4"
server_key_iv: "9a7707fa89a86c39d7392000b478ea0d"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:24.940752 29161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "1d5d5b979cab4e23b266ff10faf06f50"
format_stamp: "Formatted at 2026-08-12 00:08:24 on dist-test-slave-z8x9"
server_key: "749709948b59aabc15dc3fcb072ad7f4"
server_key_iv: "9a7707fa89a86c39d7392000b478ea0d"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:24.943162 29161 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 00:08:24.960119 29180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:24.960489 29161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 00:08:24.960611 29161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "1d5d5b979cab4e23b266ff10faf06f50"
format_stamp: "Formatted at 2026-08-12 00:08:24 on dist-test-slave-z8x9"
server_key: "749709948b59aabc15dc3fcb072ad7f4"
server_key_iv: "9a7707fa89a86c39d7392000b478ea0d"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:24.960727 29161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:08:24.986235 29161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:08:24.990204 29161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:08:24.990495 29161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:08:24.990927 29161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:08:24.991411 29161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:08:24.991462 29161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:24.991503 29161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:08:24.991535 29161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.012574 29161 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:36741
I20260812 00:08:25.012984 29161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:08:25.013644 29368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:36741 every 8 connection(s)
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493304, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:08:25.022439 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 29161
I20260812 00:08:25.022567 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:08:25.023041 7511 external_mini_cluster.cc:1428] Setting key 5ebd23bea17380963ff615e12d00fdde
I20260812 00:08:25.027030 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.013795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43909 (local address 127.7.85.193:60645)
0812 00:08:25.013975 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.014020 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.014623 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.015291 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.015296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.015658 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:08:25.015998 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.016005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.017069 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.017070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.018468 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.018473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.019204 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.019206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.019447 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.020313 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:08:25.020325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:08:25.022524 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:08:25.025151 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:08:25.025164 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:08:25.025167 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:08:25.025548 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:08:25.025797 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:08:25.025798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:08:25.025798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:08:25.025950 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:08:25.026283 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:08:25.026292 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:08:25.026483 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.026589 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.026772 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260812 00:08:25.027529 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.014118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:60645 (local address 127.7.85.254:43909)
0812 00:08:25.014225 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.014227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.014238 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:08:25.014738 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.014739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.014754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.014828 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:08:25.016125 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.016941 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.018597 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.019010 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.019098 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.022689 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:08:25.022691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:08:25.022692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:08:25.022701 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:08:25.025044 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.025646 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.025648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.025648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.025721 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.026023 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.026024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.026024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.026181 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:08:25.026253 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.027323 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.027423 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":47,"threads_started":1}
I20260812 00:08:25.027901 29371 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43909
I20260812 00:08:25.028003 29371 heartbeater.cc:464] Registering TS with master...
I20260812 00:08:25.028271 29371 heartbeater.cc:511] Master 127.7.85.254:43909 requested a full tablet report, sending...
I20260812 00:08:25.028842 29015 ts_manager.cc:194] Registered new tserver with Master: 1d5d5b979cab4e23b266ff10faf06f50 (127.7.85.193:36741)
I20260812 00:08:25.029898 29015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:60645
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:08:25.078346 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:52937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43909
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:08:25.254068 29387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:08:25.254371 29387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:08:25.254420 29387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:08:25.255846 29387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:08:25.255923 29387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:08:25.255972 29387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:08:25.257882 29387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.85.212:52937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:08:25.258191 29387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:08:25.258481 29387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:08:25.268177 29407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:25.268554 29410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:25.268177 29406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:08:25.269619 29387 server_base.cc:1034] running on GCE node
I20260812 00:08:25.269896 29387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:08:25.270237 29387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:08:25.271421 29387 hybrid_clock.cc:648] HybridClock initialized: now 1786493305271405 us; error 26 us; skew 500 ppm
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:08:25.275270 29387 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:08:25.276443 29387 webserver.cc:460] Webserver started at http://127.7.85.194:39717/ using document root <none> and password file <none>
I20260812 00:08:25.276913 29387 fs_manager.cc:359] Metadata directory not provided
I20260812 00:08:25.277053 29387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:08:25.277274 29387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:08:25.292591 29387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "a35901cdcf444581919c535fd67a0c35"
format_stamp: "Formatted at 2026-08-12 00:08:25 on dist-test-slave-z8x9"
server_key: "145fab8fa6b74ab6cba7b0a224454913"
server_key_iv: "f9245f3ba29506cb5b1aa52c87eab451"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:25.293197 29387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "a35901cdcf444581919c535fd67a0c35"
format_stamp: "Formatted at 2026-08-12 00:08:25 on dist-test-slave-z8x9"
server_key: "145fab8fa6b74ab6cba7b0a224454913"
server_key_iv: "f9245f3ba29506cb5b1aa52c87eab451"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:25.295645 29387 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:08:25.316287 29421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.316504 29387 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 00:08:25.316596 29387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "a35901cdcf444581919c535fd67a0c35"
format_stamp: "Formatted at 2026-08-12 00:08:25 on dist-test-slave-z8x9"
server_key: "145fab8fa6b74ab6cba7b0a224454913"
server_key_iv: "f9245f3ba29506cb5b1aa52c87eab451"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:25.316720 29387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:08:25.333879 29387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:08:25.337282 29387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:08:25.337711 29387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:08:25.338243 29387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:08:25.339332 29387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:08:25.339417 29387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.339461 29387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:08:25.339490 29387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.361364 29387 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:41479
I20260812 00:08:25.362105 29387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:08:25.366801 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 29387
I20260812 00:08:25.367005 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:08:25.367223 7511 external_mini_cluster.cc:1428] Setting key 3e7581a58c9d609ce18d9a880e6f6339
I20260812 00:08:25.367461 29610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:41479 every 8 connection(s)
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
I20260812 00:08:25.388638 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.363791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:56707 (local address 127.7.85.254:43909)
0812 00:08:25.363885 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.363888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.373727 (+ 9839us) server_negotiation.cc:407] Connection header received
0812 00:08:25.374890 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.374892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.374909 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.374959 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:08:25.375948 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.376883 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.378958 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.379351 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.379437 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.382999 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:08:25.383001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:08:25.383002 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:08:25.383022 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:08:25.386170 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.387068 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.387070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.387070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.387144 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.387640 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.387641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.387641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.387785 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:08:25.387845 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.388366 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.388522 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 00:08:25.388654 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.363802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43909 (local address 127.7.85.194:56707)
0812 00:08:25.373437 (+ 9635us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.373524 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.374714 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.375041 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.375046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.375496 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:08:25.375828 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.375833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.377031 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.377033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.378839 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.378844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.379477 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.379478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.379729 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.380470 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:08:25.380483 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:08:25.382803 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:08:25.386467 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:08:25.386483 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:08:25.386487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:08:25.386929 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:08:25.387216 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:08:25.387217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:08:25.387217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:08:25.387372 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:08:25.387848 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:08:25.387856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:08:25.388049 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.388167 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.388351 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9521,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
I20260812 00:08:25.390211 29611 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43909
I20260812 00:08:25.390349 29611 heartbeater.cc:464] Registering TS with master...
I20260812 00:08:25.390587 29611 heartbeater.cc:511] Master 127.7.85.254:43909 requested a full tablet report, sending...
I20260812 00:08:25.391060 29015 ts_manager.cc:194] Registered new tserver with Master: a35901cdcf444581919c535fd67a0c35 (127.7.85.194:41479)
I20260812 00:08:25.391604 29015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:56707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:08:25.417842 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:52937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43909
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:08:25.617496 29623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:08:25.617838 29623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:08:25.617880 29623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:08:25.620110 29623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:08:25.620201 29623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:08:25.620249 29623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:08:25.624011 29623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.85.212:52937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:43909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:08:25.624401 29623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:08:25.624747 29623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:08:25.635138 29632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:25.635591 29633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:08:25.635665 29638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:08:25.636436 29623 server_base.cc:1034] running on GCE node
I20260812 00:08:25.636787 29623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:08:25.637208 29623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:08:25.638439 29623 hybrid_clock.cc:648] HybridClock initialized: now 1786493305638408 us; error 49 us; skew 500 ppm
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:08:25.643385 29623 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:08:25.644620 29623 webserver.cc:460] Webserver started at http://127.7.85.195:44991/ using document root <none> and password file <none>
I20260812 00:08:25.645207 29623 fs_manager.cc:359] Metadata directory not provided
I20260812 00:08:25.645321 29623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:08:25.645493 29623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:08:25.669073 29623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3"
format_stamp: "Formatted at 2026-08-12 00:08:25 on dist-test-slave-z8x9"
server_key: "bdc690175bc4c24a63fd5dbe66d3e795"
server_key_iv: "1aa3ab10113319b8fedd3eb357cd7d31"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:25.669744 29623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3"
format_stamp: "Formatted at 2026-08-12 00:08:25 on dist-test-slave-z8x9"
server_key: "bdc690175bc4c24a63fd5dbe66d3e795"
server_key_iv: "1aa3ab10113319b8fedd3eb357cd7d31"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:25.672323 29623 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:08:25.690958 29652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.691272 29623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 00:08:25.691377 29623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3"
format_stamp: "Formatted at 2026-08-12 00:08:25 on dist-test-slave-z8x9"
server_key: "bdc690175bc4c24a63fd5dbe66d3e795"
server_key_iv: "1aa3ab10113319b8fedd3eb357cd7d31"
server_key_version: "kuduclusterkey@0"
I20260812 00:08:25.691504 29623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:08:25.702030 29623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:08:25.705322 29623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:08:25.705600 29623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:08:25.706053 29623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:08:25.706492 29623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:08:25.706560 29623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.706609 29623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:08:25.706634 29623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:08:25.734963 29623 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:43049
I20260812 00:08:25.735141 29872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:43049 every 8 connection(s)
I20260812 00:08:25.735644 29623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:08:25.736102 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 29623
I20260812 00:08:25.736248 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:08:25.736497 7511 external_mini_cluster.cc:1428] Setting key 97ecba3d71eee86049d777944cf9cdbf
Aug 12 00:08:25 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:08:25.751883 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.738095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:38457 (local address 127.7.85.254:43909)
0812 00:08:25.738145 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.738148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.738931 (+ 783us) server_negotiation.cc:407] Connection header received
0812 00:08:25.740459 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.740461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.740473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.740523 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:08:25.741923 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.742563 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.744455 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.744687 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.744756 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.748446 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:08:25.748448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:08:25.748448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:08:25.748464 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:08:25.750262 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.750797 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.750799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.750799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.750866 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.751144 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.751145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.751146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.751288 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:08:25.751363 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.751684 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.751774 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:08:25.752152 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.738106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:43909 (local address 127.7.85.195:38457)
0812 00:08:25.738650 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.738736 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.740292 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.740609 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.740613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.741240 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:08:25.741766 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.741783 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.742697 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.742699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.744358 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.744361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.744793 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.744795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.744964 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.745905 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:08:25.745937 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:08:25.748323 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:08:25.750370 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:08:25.750380 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:08:25.750383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:08:25.750656 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:08:25.750944 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:08:25.750945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:08:25.750945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:08:25.751072 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:08:25.751367 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:08:25.751374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:08:25.751536 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.751649 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.751834 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":358,"threads_started":1}
I20260812 00:08:25.752959 29878 heartbeater.cc:347] Connected to a master server at 127.7.85.254:43909
I20260812 00:08:25.753105 29878 heartbeater.cc:464] Registering TS with master...
I20260812 00:08:25.753381 29878 heartbeater.cc:511] Master 127.7.85.254:43909 requested a full tablet report, sending...
I20260812 00:08:25.753917 29015 ts_manager.cc:194] Registered new tserver with Master: 2fdabb1684dc4b969cbee2dafd8ff1e3 (127.7.85.195:43049)
I20260812 00:08:25.754482 29015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:38457
I20260812 00:08:25.760999 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:08:25.771329 29378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.763523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.7.85.254:43909)
0812 00:08:25.763615 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.763618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.763928 (+ 310us) server_negotiation.cc:407] Connection header received
0812 00:08:25.764016 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.764017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.764028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.764105 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:08:25.764884 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.765522 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.766310 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.766578 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.766656 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.768233 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:08:25.768234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:08:25.768235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:08:25.768243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:08:25.770006 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.770453 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.770454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.770455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.770522 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:08:25.770742 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:08:25.770743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:08:25.770744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:08:25.770886 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:08:25.770995 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.771128 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.771213 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 00:08:25.773707 29015 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:08:25.776865 29015 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:08:25.786903 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.780484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:41479 (local address 127.0.0.1:56374)
0812 00:08:25.781152 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.781263 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.781541 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.782471 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.782475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.782478 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:08:25.782573 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.782575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.783591 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.783592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.784436 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.784439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.786427 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.786429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.786584 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.786605 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.786692 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.786765 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":169,"threads_started":1}
I20260812 00:08:25.787706 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.780674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.7.85.194:41479)
0812 00:08:25.782202 (+ 1528us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.782206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.782214 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:08:25.782259 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.782261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.782274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.782466 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:08:25.782636 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.783481 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.784657 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.786264 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.787178 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.787399 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.787433 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.787497 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1476,"thread_start_us":258,"threads_started":1}
I20260812 00:08:25.788756 29502 tablet_service.cc:1467] Processing CreateTablet for tablet 14894f0b9c8c467b9acb6322e495ba31 (DEFAULT_TABLE table=test-table [id=70fcc03102b048ada2187bfe95c841b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:08:25.789306 29502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14894f0b9c8c467b9acb6322e495ba31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:08:25.792965 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.780894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:43049 (local address 127.0.0.1:56694)
0812 00:08:25.785456 (+ 4562us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.785478 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.785767 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.786569 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.786571 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.786574 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:08:25.786668 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.786670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.789214 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.789216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.790724 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.790727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.792600 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.792601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.792717 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.792734 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.792831 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.792878 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4490,"thread_start_us":423,"threads_started":1}
I20260812 00:08:25.793251 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.780620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:36741 (local address 127.0.0.1:38778)
0812 00:08:25.781699 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.781709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.781958 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.786480 (+ 4522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.786494 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.786497 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:08:25.786565 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.786567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.788722 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.788724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.790723 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.790727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.792655 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.792657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.792808 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.792813 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.792894 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.793072 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":207,"threads_started":1}
I20260812 00:08:25.793752 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.781622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.7.85.195:43049)
0812 00:08:25.782137 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.782143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.785862 (+ 3719us) server_negotiation.cc:407] Connection header received
0812 00:08:25.785987 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.785991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.786014 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.786397 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:08:25.786769 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.789004 (+ 2235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.790947 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.792213 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.792313 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.792539 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.793347 (+ 808us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.793556 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
I20260812 00:08:25.794090 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.781079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38778 (local address 127.7.85.193:36741)
0812 00:08:25.785468 (+ 4389us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.785474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.785495 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:08:25.785556 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.785558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.785591 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.785892 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:08:25.786714 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.788552 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.791071 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.792456 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.793668 (+ 1212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.793807 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.793861 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.793923 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4330,"thread_start_us":239,"threads_started":1}
I20260812 00:08:25.795204 29264 tablet_service.cc:1467] Processing CreateTablet for tablet 14894f0b9c8c467b9acb6322e495ba31 (DEFAULT_TABLE table=test-table [id=70fcc03102b048ada2187bfe95c841b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:08:25.795679 29264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14894f0b9c8c467b9acb6322e495ba31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:08:25.795755 29739 tablet_service.cc:1467] Processing CreateTablet for tablet 14894f0b9c8c467b9acb6322e495ba31 (DEFAULT_TABLE table=test-table [id=70fcc03102b048ada2187bfe95c841b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:08:25.796120 29739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14894f0b9c8c467b9acb6322e495ba31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:08:25.801921 29921 tablet_bootstrap.cc:492] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Bootstrap starting.
I20260812 00:08:25.802696 29922 tablet_bootstrap.cc:492] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50: Bootstrap starting.
I20260812 00:08:25.802701 29924 tablet_bootstrap.cc:492] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Bootstrap starting.
I20260812 00:08:25.804535 29922 tablet_bootstrap.cc:654] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50: Neither blocks nor log segments found. Creating new log.
I20260812 00:08:25.804962 29924 tablet_bootstrap.cc:654] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Neither blocks nor log segments found. Creating new log.
I20260812 00:08:25.805150 29921 tablet_bootstrap.cc:654] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Neither blocks nor log segments found. Creating new log.
I20260812 00:08:25.805668 29922 log.cc:824] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50: Log is configured to *not* fsync() on all Append() calls
I20260812 00:08:25.805826 29924 log.cc:824] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:08:25.806398 29921 log.cc:824] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Log is configured to *not* fsync() on all Append() calls
I20260812 00:08:25.806972 29922 tablet_bootstrap.cc:492] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50: No bootstrap required, opened a new log
I20260812 00:08:25.807057 29922 ts_tablet_manager.cc:1397] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260812 00:08:25.807185 29924 tablet_bootstrap.cc:492] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: No bootstrap required, opened a new log
I20260812 00:08:25.807317 29924 ts_tablet_manager.cc:1397] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 00:08:25.808265 29921 tablet_bootstrap.cc:492] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: No bootstrap required, opened a new log
I20260812 00:08:25.808369 29921 ts_tablet_manager.cc:1397] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260812 00:08:25.812057 29924 raft_consensus.cc:348] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.812394 29924 raft_consensus.cc:374] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:08:25.812453 29924 raft_consensus.cc:729] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdabb1684dc4b969cbee2dafd8ff1e3, State: Initialized, Role: FOLLOWER
I20260812 00:08:25.812597 29924 consensus_queue.cc:260] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.813139 29924 ts_tablet_manager.cc:1428] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260812 00:08:25.813414 29878 heartbeater.cc:503] Master 127.7.85.254:43909 was elected leader, sending a full tablet report...
I20260812 00:08:25.814543 29922 raft_consensus.cc:348] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.814749 29922 raft_consensus.cc:374] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:08:25.814795 29922 raft_consensus.cc:729] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5d5b979cab4e23b266ff10faf06f50, State: Initialized, Role: FOLLOWER
I20260812 00:08:25.814747 29921 raft_consensus.cc:348] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.814920 29922 consensus_queue.cc:260] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.815018 29921 raft_consensus.cc:374] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:08:25.815093 29921 raft_consensus.cc:729] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35901cdcf444581919c535fd67a0c35, State: Initialized, Role: FOLLOWER
I20260812 00:08:25.815301 29921 consensus_queue.cc:260] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.815448 29922 ts_tablet_manager.cc:1428] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20260812 00:08:25.815613 29371 heartbeater.cc:503] Master 127.7.85.254:43909 was elected leader, sending a full tablet report...
I20260812 00:08:25.815647 29921 ts_tablet_manager.cc:1428] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 00:08:25.815835 29611 heartbeater.cc:503] Master 127.7.85.254:43909 was elected leader, sending a full tablet report...
W20260812 00:08:25.863426 29613 tablet.cc:2369] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:08:25.915853 29939 raft_consensus.cc:482] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:08:25.916029 29939 raft_consensus.cc:504] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.916508 29939 leader_election.cc:288] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d5d5b979cab4e23b266ff10faf06f50 (127.7.85.193:36741), 2fdabb1684dc4b969cbee2dafd8ff1e3 (127.7.85.195:43049)
I20260812 00:08:25.921214 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.916921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:59249 (local address 127.7.85.193:36741)
0812 00:08:25.916973 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.916976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.917222 (+ 246us) server_negotiation.cc:407] Connection header received
0812 00:08:25.917371 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.917372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.917387 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.917420 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:08:25.917653 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.918407 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.920002 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.920692 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.920743 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.920753 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.921068 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.921116 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:08:25.921828 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.916697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:36741 (local address 127.7.85.194:59249)
0812 00:08:25.917094 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.917106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.917258 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.917494 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.917495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.917499 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:08:25.917572 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.917574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.918504 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.918505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.919868 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.919872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.920842 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.920843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.920947 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.920951 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.921586 (+ 635us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.921636 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":72,"threads_started":1}
I20260812 00:08:25.922981 29296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14894f0b9c8c467b9acb6322e495ba31" candidate_uuid: "a35901cdcf444581919c535fd67a0c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d5b979cab4e23b266ff10faf06f50" is_pre_election: true
I20260812 00:08:25.923170 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.916995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:55403 (local address 127.7.85.195:43049)
0812 00:08:25.917039 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.917041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.918445 (+ 1404us) server_negotiation.cc:407] Connection header received
0812 00:08:25.918565 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.918567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.918581 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.918610 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:08:25.918988 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.919746 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.921463 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.922644 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.922692 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.922704 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.923035 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.923087 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:08:25.923206 29296 raft_consensus.cc:2454] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35901cdcf444581919c535fd67a0c35 in term 0.
I20260812 00:08:25.923319 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.916699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:43049 (local address 127.7.85.194:55403)
0812 00:08:25.918359 (+ 1660us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:25.918376 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:08:25.918473 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:08:25.918672 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:08:25.918673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:08:25.918674 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:08:25.918889 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.918892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.919938 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.919939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.921360 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:08:25.921364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.922808 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.922809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:08:25.922941 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.922944 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:08:25.923015 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 00:08:25.923060 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1616,"mutex_wait_us":12,"thread_start_us":180,"threads_started":1}
I20260812 00:08:25.923710 29777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14894f0b9c8c467b9acb6322e495ba31" candidate_uuid: "a35901cdcf444581919c535fd67a0c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" is_pre_election: true
I20260812 00:08:25.923853 29777 raft_consensus.cc:2454] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a35901cdcf444581919c535fd67a0c35 in term 0.
I20260812 00:08:25.924072 29432 leader_election.cc:302] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5d5b979cab4e23b266ff10faf06f50, a35901cdcf444581919c535fd67a0c35; no voters:
I20260812 00:08:25.924240 29939 raft_consensus.cc:2781] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:08:25.924293 29939 raft_consensus.cc:482] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:08:25.924316 29939 raft_consensus.cc:3037] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:08:25.925446 29939 raft_consensus.cc:504] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.925594 29939 leader_election.cc:288] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [CANDIDATE]: Term 1 election: Requested vote from peers 1d5d5b979cab4e23b266ff10faf06f50 (127.7.85.193:36741), 2fdabb1684dc4b969cbee2dafd8ff1e3 (127.7.85.195:43049)
I20260812 00:08:25.925748 29296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14894f0b9c8c467b9acb6322e495ba31" candidate_uuid: "a35901cdcf444581919c535fd67a0c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5d5b979cab4e23b266ff10faf06f50"
I20260812 00:08:25.925808 29777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14894f0b9c8c467b9acb6322e495ba31" candidate_uuid: "a35901cdcf444581919c535fd67a0c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3"
I20260812 00:08:25.925838 29296 raft_consensus.cc:3037] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:08:25.925879 29777 raft_consensus.cc:3037] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:08:25.926962 29296 raft_consensus.cc:2454] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a35901cdcf444581919c535fd67a0c35 in term 1.
I20260812 00:08:25.927170 29432 leader_election.cc:302] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5d5b979cab4e23b266ff10faf06f50, a35901cdcf444581919c535fd67a0c35; no voters:
I20260812 00:08:25.927300 29777 raft_consensus.cc:2454] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a35901cdcf444581919c535fd67a0c35 in term 1.
I20260812 00:08:25.927516 29939 raft_consensus.cc:2781] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:08:25.927723 29939 raft_consensus.cc:686] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 1 LEADER]: Becoming Leader. State: Replica: a35901cdcf444581919c535fd67a0c35, State: Running, Role: LEADER
I20260812 00:08:25.927838 29939 consensus_queue.cc:237] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } }
I20260812 00:08:25.928643 29015 catalog_manager.cc:5495] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 reported cstate change: term changed from 0 to 1, leader changed from <none> to a35901cdcf444581919c535fd67a0c35 (127.7.85.194). New cstate: current_term: 1 leader_uuid: "a35901cdcf444581919c535fd67a0c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a35901cdcf444581919c535fd67a0c35" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 41479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 } health_report { overall_health: UNKNOWN } } }
I20260812 00:08:25.942579 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:25.938911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.7.85.194:41479)
0812 00:08:25.939082 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 00:08:25.939085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:08:25.939093 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:08:25.939135 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:08:25.939136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:08:25.939160 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:08:25.939244 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:08:25.939439 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.940153 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.941340 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:08:25.941608 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:08:25.941836 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:08:25.942090 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:08:25.942239 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:08:25.942349 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 00:08:25.942431 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 00:08:25.942457 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260812 00:08:25.946504 29777 raft_consensus.cc:1261] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 1 FOLLOWER]: Refusing update from remote peer a35901cdcf444581919c535fd67a0c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:08:25.946904 29939 consensus_queue.cc:1035] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdabb1684dc4b969cbee2dafd8ff1e3" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 43049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:08:25.949301 29296 raft_consensus.cc:1261] T 14894f0b9c8c467b9acb6322e495ba31 P 1d5d5b979cab4e23b266ff10faf06f50 [term 1 FOLLOWER]: Refusing update from remote peer a35901cdcf444581919c535fd67a0c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:08:25.949591 29939 consensus_queue.cc:1035] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5d5b979cab4e23b266ff10faf06f50" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 36741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:08:25.952922 29946 mvcc.cc:205] Tried to move back new op lower bound from 7317476581153038336 to 7317476581080981504. Current Snapshot: MvccSnapshot[applied={T|T < 7317476581153038336}]
I20260812 00:08:25.953193 29948 mvcc.cc:205] Tried to move back new op lower bound from 7317476581153038336 to 7317476581080981504. Current Snapshot: MvccSnapshot[applied={T|T < 7317476581153038336}]
I20260812 00:08:25.955602 29952 mvcc.cc:205] Tried to move back new op lower bound from 7317476581153038336 to 7317476581080981504. Current Snapshot: MvccSnapshot[applied={T|T < 7317476581153038336}]
I20260812 00:08:25.959162 29015 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:08:25.959292 29015 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:08:25.960083 29015 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6392fda2fb404a3e96732417430872eb: Sending DeleteTablet for 3 replicas of tablet 14894f0b9c8c467b9acb6322e495ba31
I20260812 00:08:25.960525 29264 tablet_service.cc:1514] Processing DeleteTablet for tablet 14894f0b9c8c467b9acb6322e495ba31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:08:25 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:38778
I20260812 00:08:25.961078 29739 tablet_service.cc:1514] Processing DeleteTablet for tablet 14894f0b9c8c467b9acb6322e495ba31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:08:25 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:56694
I20260812 00:08:25.961632 29502 tablet_service.cc:1514] Processing DeleteTablet for tablet 14894f0b9c8c467b9acb6322e495ba31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:08:25 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:56374
I20260812 00:08:25.962318 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 29161
I20260812 00:08:25.974014 29961 tablet_replica.cc:330] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: stopping tablet replica
I20260812 00:08:25.974238 29961 raft_consensus.cc:2229] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:08:25.974439 29961 raft_consensus.cc:2258] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:08:25.974696 29963 tablet_replica.cc:330] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: stopping tablet replica
I20260812 00:08:25.974856 29963 raft_consensus.cc:2229] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:08:25.975026 29963 raft_consensus.cc:2258] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:08:26.062654 29963 ts_tablet_manager.cc:1905] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:08:26.063113 29961 ts_tablet_manager.cc:1905] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:08:26.065522 29961 ts_tablet_manager.cc:1918] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:08:26.065642 29961 log.cc:1196] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/14894f0b9c8c467b9acb6322e495ba31
I20260812 00:08:26.065974 29963 ts_tablet_manager.cc:1918] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:08:26.066007 29961 ts_tablet_manager.cc:1939] T 14894f0b9c8c467b9acb6322e495ba31 P 2fdabb1684dc4b969cbee2dafd8ff1e3: Deleting consensus metadata
I20260812 00:08:26.066075 29963 log.cc:1196] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/14894f0b9c8c467b9acb6322e495ba31
I20260812 00:08:26.066352 29963 ts_tablet_manager.cc:1939] T 14894f0b9c8c467b9acb6322e495ba31 P a35901cdcf444581919c535fd67a0c35: Deleting consensus metadata
I20260812 00:08:26.066723 28991 catalog_manager.cc:4841] TS 2fdabb1684dc4b969cbee2dafd8ff1e3 (127.7.85.195:43049): tablet 14894f0b9c8c467b9acb6322e495ba31 (table test-table [id=70fcc03102b048ada2187bfe95c841b0]) successfully deleted
I20260812 00:08:26.067090 28991 catalog_manager.cc:4841] TS a35901cdcf444581919c535fd67a0c35 (127.7.85.194:41479): tablet 14894f0b9c8c467b9acb6322e495ba31 (table test-table [id=70fcc03102b048ada2187bfe95c841b0]) successfully deleted
W20260812 00:08:26.076692 28993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:36741 (error 108)
I20260812 00:08:26.077574 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 29387
I20260812 00:08:26.099062 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:08:26.098649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:36741 (local address 127.0.0.1:38780)
0812 00:08:26.098800 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 00:08:26.098914 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:36741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
Aug 12 00:08:26 dist-test-slave-z8x9 krb5kdc[28153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
W20260812 00:08:26.099283 28993 catalog_manager.cc:4584] TS 1d5d5b979cab4e23b266ff10faf06f50 (127.7.85.193:36741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14894f0b9c8c467b9acb6322e495ba31: Network error: Client connection negotiation failed: client connection to 127.7.85.193:36741: connect: Connection refused (error 111)
I20260812 00:08:26.106341 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 29623
I20260812 00:08:26.133227 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 28959
I20260812 00:08:26.155398 7511 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 00:08:26.513330 7511 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 00:08:26.513458 7511 mini_ranger.cc:66] Stopping Ranger...
I20260812 00:08:26.872632 7511 mini_ranger.cc:68] Stopped Ranger
2026-08-12 00:08:26.872 UTC [28207] LOG: received smart shutdown request
2026-08-12 00:08:26.877 UTC [28207] LOG: background worker "logical replication launcher" (PID 28223) exited with exit code 1
2026-08-12 00:08:26.878 UTC [28216] LOG: shutting down
2026-08-12 00:08:27.037 UTC [28207] LOG: database system is shut down
2026-08-12T00:08:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (55640 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:08:27 dist-test-slave-z8x9 krb5kdc[29979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:08:27 dist-test-slave-z8x9 krb5kdc[29979](info): set up 2 sockets
Aug 12 00:08:27 dist-test-slave-z8x9 krb5kdc[29979](info): commencing operation
krb5kdc: starting...
W20260812 00:08:28.552421 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.372s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:08:28 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:08:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/rangeradmin_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/rangeradmin_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/rangerlookup_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/rangerlookup_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
I20260812 00:08:28.749269 7511 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskaxXthI/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/postgres -l logfile start
2026-08-12 00:08:31.759 UTC [30036] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 00:08:31.759 UTC [30036] LOG: listening on IPv4 address "127.7.85.212", port 57969
2026-08-12 00:08:31.763 UTC [30036] LOG: listening on Unix socket "/tmp/.s.PGSQL.57969"
2026-08-12 00:08:31.789 UTC [30042] LOG: database system was shut down at 2026-08-12 00:08:29 UTC
2026-08-12 00:08:31.794 UTC [30036] LOG: database system is ready to accept connections
I20260812 00:08:33.129769 7511 mini_postgres.cc:96] Postgres bound to 57969
2026-08-12 00:08:33.137 UTC [30058] FATAL: database "slave" does not exist
127.7.85.212:57969 - accepting connections
I20260812 00:08:33.138579 7511 mini_ranger.cc:161] Starting Ranger...
I20260812 00:08:33.157939 7511 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 00:08:33.818482 7511 mini_ranger.cc:87] Created ranger Postgres database
I20260812 00:08:33.818598 7511 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-admin
2026-08-12 00:08:34,128 [I] DB FLAVOR :POSTGRES
2026-08-12 00:08:34,128 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:08:34,129 [I] Checking connection..
2026-08-12 00:08:34,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 00:08:34,421 [I] Checking connection passed.
2026-08-12 00:08:34,422 [I] --------- Verifying version history table ---------
2026-08-12 00:08:34,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:08:34,713 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:08:34,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:08:35,004 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:08:35,004 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 00:08:35,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 00:08:35.286 UTC [30166] WARNING: there is no transaction in progress
2026-08-12 00:08:35,294 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 00:08:35,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:08:35,582 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 00:08:35,582 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:08:35,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 00:08:35,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 00:08:36,194 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z8x9.c.gcp-upstream.internal','N') ;"
2026-08-12 00:08:36,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:08:36,791 [I] Table x_portal_user does not exist in database ranger
2026-08-12 00:08:36,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:08:37,062 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 00:08:37,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:08:37,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:08:37,645 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 00:08:37,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 00:08:38.917 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:38.943 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:38.967 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:38.996 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.020 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.045 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.123 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.133 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.154 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.169 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.184 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.203 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.218 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.228 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.240 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:39.253 UTC [30372] WARNING: there is no transaction in progress
2026-08-12 00:08:40,193 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 00:08:40,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 00:08:40,500 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 00:08:40,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:08:40,773 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 00:08:40,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:08:41,062 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 00:08:41,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:08:41,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:08:41,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z8x9.c.gcp-upstream.internal';"
2026-08-12 00:08:41,892 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 00:08:41,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 00:08:42,169 [I] DB_PATCHES have already been applied
I20260812 00:08:42.176543 7511 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-admin:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskaxXthI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 00:08:42.176627 7511 mini_ranger.cc:193] Using host: 127.7.85.212
I20260812 00:08:42.181020 7511 mini_ranger.cc:227] Ranger admin URL: http://127.7.85.212:60231
Aug 12, 2026 12:08:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:08:42 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:08:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 12:08:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 12:08:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 12:08:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 12:08:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 00:08:43.716743 7511 mini_ranger.cc:160] Time spent starting Ranger: real 10.578s user 0.000s sys 0.015s
Aug 12, 2026 12:09:05 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 12:09:05 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 12:09:05 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 12:09:05 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 12:09:06 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 00:09:06.753620 7511 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.85.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.85.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/rangerkms_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.85.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/rangerkms_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.85.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.85.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.85.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/HTTP_127.7.85.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 00:09:06 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493346, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 00:09:06.866108 7511 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 00:09:06.878317 7511 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 00:09:08.051234 7511 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 00:09:08.051340 7511 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms
I20260812 00:09:08.051360 7511 mini_ranger_kms.cc:227] Using postgres at 127.7.85.212:57969
2026-08-12 00:09:08,242 [I] DB FLAVOR :POSTGRES
2026-08-12 00:09:08,242 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:09:08,242 [I] Checking connection
2026-08-12 00:09:08,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 00:09:08,533 [I] connection success
2026-08-12 00:09:08,534 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 00:09:08,534 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 00:09:08,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 00:09:08,817 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 00:09:08,817 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:09:08,818 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 00:09:08,818 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.85.212:57969/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 00:09:09,119 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 00:09:09.668838 7511 mini_ranger_kms.cc:310] Created kms service
I20260812 00:09:09.872509 7511 mini_ranger_kms.cc:326] Created kudu user
I20260812 00:09:09.922441 7511 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 00:09:09 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493346, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:10.459378 7511 mini_ranger_kms.cc:384] Added ranger policy
I20260812 00:09:10.459511 7511 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms:/tmp/dist-test-taskaxXthI/build/release/bin/postgresql.jar:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskaxXthI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 00:09:10.459532 7511 mini_ranger_kms.cc:240] Using host: 127.7.85.212
I20260812 00:09:10.465365 7511 mini_ranger_kms.cc:276] Ranger KMS PID: 30707
I20260812 00:09:10.465601 7511 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.85.212:54643
Aug 12, 2026 12:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 12:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 12:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 12:09:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 00:09:12 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786493352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.85.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 00:09:13 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.85.212@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:13.466779 7511 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.601s user 0.000s sys 0.174s
I20260812 00:09:13.466889 7511 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 00:09:13.466935 7511 mini_ranger_kms.cc:401] 127.7.85.212:54643/kms/v1/keys
Aug 12 00:09:13 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493352, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.85.212@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
Aug 12 00:09:16 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:09:16.288403 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:34823
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:34823
--ranger_config_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:16.404315 30802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:16.404585 30802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:16.404627 30802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:16.406010 30802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:16.406076 30802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:16.406103 30802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:16.406152 30802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:16.406180 30802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:16.407845 30802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.85.212:54643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:34823
--ranger_config_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:34823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:16.408138 30802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:16.408391 30802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:16.415115 30813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:16.415235 30816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:16.415119 30818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:16.415203 30802 server_base.cc:1034] running on GCE node
I20260812 00:09:16.415934 30802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:16.416299 30802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:16.417474 30802 hybrid_clock.cc:648] HybridClock initialized: now 1786493356417447 us; error 48 us; skew 500 ppm
Aug 12 00:09:16 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:16.420625 30802 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:16.421221 30802 webserver.cc:460] Webserver started at http://127.7.85.254:37881/ using document root <none> and password file <none>
I20260812 00:09:16.421494 30802 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:16.421537 30802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:16.421684 30802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:09:16 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:16.481139 30802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "a4f9ca5d39de4340afda566a1908e087"
format_stamp: "Formatted at 2026-08-12 00:09:16 on dist-test-slave-z8x9"
server_key: "202df43d7c3ba834709181ed613841f8"
server_key_iv: "afd81d8de94f0f0c4dd535497c7d8cbd"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:16.481607 30802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "a4f9ca5d39de4340afda566a1908e087"
format_stamp: "Formatted at 2026-08-12 00:09:16 on dist-test-slave-z8x9"
server_key: "202df43d7c3ba834709181ed613841f8"
server_key_iv: "afd81d8de94f0f0c4dd535497c7d8cbd"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:16.483292 30802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:09:16.503603 30832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:16.504190 30802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:16.504369 30802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "a4f9ca5d39de4340afda566a1908e087"
format_stamp: "Formatted at 2026-08-12 00:09:16 on dist-test-slave-z8x9"
server_key: "202df43d7c3ba834709181ed613841f8"
server_key_iv: "afd81d8de94f0f0c4dd535497c7d8cbd"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:16.504635 30802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:16.530684 30802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:16.533020 30802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:16.533258 30802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:16.538904 30802 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:34823
I20260812 00:09:16.538937 30931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:34823 every 8 connection(s)
I20260812 00:09:16.539371 30802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:16.540136 30934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:16.542647 30934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Bootstrap starting.
I20260812 00:09:16.543311 30934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:16.543640 30934 log.cc:824] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:16.544377 30934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: No bootstrap required, opened a new log
I20260812 00:09:16.546559 30934 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:16.546762 30934 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:16.546793 30934 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f9ca5d39de4340afda566a1908e087, State: Initialized, Role: FOLLOWER
I20260812 00:09:16.547041 30934 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:16.547134 30934 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:16.547159 30934 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:16.547230 30934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:16.547887 30934 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:16.547989 30934 leader_election.cc:302] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4f9ca5d39de4340afda566a1908e087; no voters:
I20260812 00:09:16.548218 30934 leader_election.cc:288] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:09:16.548384 30938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:16.548475 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 30802
I20260812 00:09:16.548591 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:09:16.548643 30934 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:16.548717 30938 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 LEADER]: Becoming Leader. State: Replica: a4f9ca5d39de4340afda566a1908e087, State: Running, Role: LEADER
I20260812 00:09:16.548815 7511 external_mini_cluster.cc:1428] Setting key 0a07de175611821e5abbabc74b126bd2
I20260812 00:09:16.548862 30938 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:16.549511 30940 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4f9ca5d39de4340afda566a1908e087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } } }
I20260812 00:09:16.549749 30940 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:16.549501 30938 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4f9ca5d39de4340afda566a1908e087. Latest consensus state: current_term: 1 leader_uuid: "a4f9ca5d39de4340afda566a1908e087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } } }
I20260812 00:09:16.550066 30938 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:16.551105 30934 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 00:09:16.555161 30955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:16.556212 30955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:16.558547 30955 catalog_manager.cc:1347] Generated new cluster ID: 4d3c94fea35944a18540cf637bde7384
I20260812 00:09:16.558686 30955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:09:16 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:16.566862 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:16.549604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.7.85.254:34823)
0812 00:09:16.554103 (+ 4499us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:16.554118 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:16.554180 (+ 62us) server_negotiation.cc:407] Connection header received
0812 00:09:16.554429 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:16.554437 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:16.554560 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:16.554747 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:16.555781 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:16.557454 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:16.558603 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:16.559197 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:16.562298 (+ 3101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:16.562425 (+ 127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:16.562429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:16.562430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:16.562440 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:16.564446 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:16.564957 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:16.564961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:16.564961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:16.565102 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:16.565389 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:16.565390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:16.565390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:16.565787 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:16.566378 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:16.566467 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 00:09:16.566610 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4312,"thread_start_us":96,"threads_started":1}
I20260812 00:09:16.571499 30955 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:09:16.572333 30955 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:16.577502 30955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Generated new TSK 0
I20260812 00:09:16.577762 30955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:09:17.341764 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34823
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:17.465013 31011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:17.465345 31011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:17.465387 31011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:17.467166 31011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:17.467231 31011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:17.467270 31011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:09:17.468899 31011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.85.212:54643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:17.469169 31011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:17.469502 31011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:17.475016 31020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:17.475025 31022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:17.475102 31025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:17.475322 31011 server_base.cc:1034] running on GCE node
I20260812 00:09:17.475551 31011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:17.475872 31011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:17.477109 31011 hybrid_clock.cc:648] HybridClock initialized: now 1786493357477090 us; error 35 us; skew 500 ppm
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:17.480785 31011 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:09:17.481526 31011 webserver.cc:460] Webserver started at http://127.7.85.193:45865/ using document root <none> and password file <none>
I20260812 00:09:17.481928 31011 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:17.481982 31011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:17.482128 31011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:17.496032 31011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "3f728f9d4e924aa1bde0f8b11f456eba"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "8e1f3cb530b6adddd717ed652c352a0b"
server_key_iv: "535ef70b56697ca4f3d2590b47dc2f50"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:17.496634 31011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "3f728f9d4e924aa1bde0f8b11f456eba"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "8e1f3cb530b6adddd717ed652c352a0b"
server_key_iv: "535ef70b56697ca4f3d2590b47dc2f50"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:17.498891 31011 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 00:09:17.513679 31040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:17.513931 31011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:09:17.514070 31011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "3f728f9d4e924aa1bde0f8b11f456eba"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "8e1f3cb530b6adddd717ed652c352a0b"
server_key_iv: "535ef70b56697ca4f3d2590b47dc2f50"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:17.514196 31011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:17.528594 31011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:17.531517 31011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:17.531793 31011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:17.532106 31011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:17.532816 31011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:17.533077 31011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:17.533412 31011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:17.533614 31011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:17.541157 31011 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:39631
I20260812 00:09:17.541731 31011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:09:17.542043 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 31011
I20260812 00:09:17.542176 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:09:17.542444 7511 external_mini_cluster.cc:1428] Setting key a435169f1a9c87f7fd3dc74f061f0021
I20260812 00:09:17.559427 31232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:39631 every 8 connection(s)
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:09:17.568619 31235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:17.543264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34823 (local address 127.7.85.193:41151)
0812 00:09:17.543638 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:17.543691 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:17.545127 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:17.553681 (+ 8554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:17.553688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:17.554014 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:17.554374 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:17.554382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.555603 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.555604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:17.557119 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:17.557123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.557915 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.557916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:17.558092 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:17.562280 (+ 4188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:17.562295 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:17.564274 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:17.566601 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:17.566611 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:17.566614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:17.566955 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:17.567204 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:17.567205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:17.567205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:17.567357 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:17.567692 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:17.567700 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:17.567899 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 00:09:17.568162 (+ 263us) client_negotiation.cc:240] Negotiation successful
0812 00:09:17.568364 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":99,"threads_started":1}
I20260812 00:09:17.569228 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:17.545316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:41151 (local address 127.7.85.254:34823)
0812 00:09:17.553312 (+ 7996us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:17.553315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:17.553332 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:09:17.553368 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:17.553369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:17.553385 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:17.553505 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:17.554533 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.555455 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.557447 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.557776 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.558875 (+ 1099us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:17.564433 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:17.564436 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:17.564436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:17.564446 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:17.566488 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:17.567056 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:17.567057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:17.567057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:17.567127 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:17.567444 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:17.567445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:17.567445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:17.567607 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:17.568933 (+ 1326us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:17.568983 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 00:09:17.569082 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7954,"thread_start_us":100,"threads_started":1}
I20260812 00:09:17.569839 31233 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34823
I20260812 00:09:17.569941 31233 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
I20260812 00:09:17.570148 31233 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:17.570719 30868 ts_manager.cc:194] Registered new tserver with Master: 3f728f9d4e924aa1bde0f8b11f456eba (127.7.85.193:39631)
I20260812 00:09:17.571623 30868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:41151
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:09:17.593865 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34823
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:17.714555 31241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:17.714943 31241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:17.715023 31241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:17.716981 31241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:17.717065 31241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:17.717109 31241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:09:17.719493 31241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.85.212:54643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:17.719766 31241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:17.720104 31241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:17.726912 31258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:17.727028 31257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:17.727089 31261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:17.728039 31241 server_base.cc:1034] running on GCE node
I20260812 00:09:17.728324 31241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:17.728626 31241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:17.729808 31241 hybrid_clock.cc:648] HybridClock initialized: now 1786493357729784 us; error 38 us; skew 500 ppm
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:17.734233 31241 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:09:17.734942 31241 webserver.cc:460] Webserver started at http://127.7.85.194:37699/ using document root <none> and password file <none>
I20260812 00:09:17.735270 31241 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:17.735344 31241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:17.735502 31241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:17.758615 31241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "cc9071f89f4a4c78995fcab60d41ebb3"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "e9059d13402017ad08fa69b820769960"
server_key_iv: "b10ac97dda2d35810665b219ba4991f1"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:17.759106 31241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "cc9071f89f4a4c78995fcab60d41ebb3"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "e9059d13402017ad08fa69b820769960"
server_key_iv: "b10ac97dda2d35810665b219ba4991f1"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:17.762739 31241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:09:17.778384 31271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:17.778681 31241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:09:17.780277 31241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "cc9071f89f4a4c78995fcab60d41ebb3"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "e9059d13402017ad08fa69b820769960"
server_key_iv: "b10ac97dda2d35810665b219ba4991f1"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:17.780409 31241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:17.815001 31241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:17.818251 31241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:17.818534 31241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:17.818873 31241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:17.819387 31241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:17.819499 31241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:17.819590 31241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:17.819666 31241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:17.828161 31241 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:43873
I20260812 00:09:17.828830 31241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:09:17.829497 31464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:43873 every 8 connection(s)
I20260812 00:09:17.833060 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 31241
I20260812 00:09:17.833513 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:09:17.834080 7511 external_mini_cluster.cc:1428] Setting key c32fb7396a0a3d8722d043920a5cb34a
Aug 12 00:09:17 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:17.846520 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:17.830610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:46563 (local address 127.7.85.254:34823)
0812 00:09:17.830679 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:17.830681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:17.830981 (+ 300us) server_negotiation.cc:407] Connection header received
0812 00:09:17.831752 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:17.831754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:17.831790 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:17.831827 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:17.832808 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.833684 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.838898 (+ 5214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.839289 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.839343 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:17.842586 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:17.842588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:17.842588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:17.842598 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:17.844713 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:17.845314 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:17.845316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:17.845316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:17.845384 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:17.845681 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:17.845682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:17.845683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:17.845851 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:17.845907 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:17.846312 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 00:09:17.846414 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 00:09:17.847357 31468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:17.830615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34823 (local address 127.7.85.194:46563)
0812 00:09:17.830838 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:17.830887 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:17.831604 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:17.832048 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:17.832053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:17.832349 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:17.832662 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:17.832668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.833948 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.833949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:17.838720 (+ 4771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:17.838727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:17.839477 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:17.839478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:17.839649 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:17.840420 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:17.840432 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:17.842440 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:17.844844 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:17.844853 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:17.844856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:17.845215 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:17.845457 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:17.845458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:17.845458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:17.845604 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:17.845952 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:17.845960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:17.846145 (+ 185us) client_negotiation.cc:769] Sending connection context
0812 00:09:17.846920 (+ 775us) client_negotiation.cc:240] Negotiation successful
0812 00:09:17.847107 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":129,"threads_started":1}
I20260812 00:09:17.848084 31465 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34823
I20260812 00:09:17.848202 31465 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:17.848439 31465 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:17.848819 30868 ts_manager.cc:194] Registered new tserver with Master: cc9071f89f4a4c78995fcab60d41ebb3 (127.7.85.194:43873)
I20260812 00:09:17.849375 30868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:46563
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:09:17.889439 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34823
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:17.991719 31472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:17.991984 31472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:17.992026 31472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:17.993264 31472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:17.993333 31472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:17.993381 31472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:09:17.994896 31472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.85.212:54643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:34823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:17.995172 31472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:17.995455 31472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:18.002707 31487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:18.003006 31484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:18.003252 31481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:18.003580 31472 server_base.cc:1034] running on GCE node
I20260812 00:09:18.003895 31472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:18.004207 31472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:18.007485 31472 hybrid_clock.cc:648] HybridClock initialized: now 1786493358007456 us; error 51 us; skew 500 ppm
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:18.011420 31472 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:09:18.012115 31472 webserver.cc:460] Webserver started at http://127.7.85.195:46861/ using document root <none> and password file <none>
I20260812 00:09:18.012427 31472 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:18.012512 31472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:18.012679 31472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:18.031383 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "303fe15ebd81427ea3b91fb527b2eeef"
format_stamp: "Formatted at 2026-08-12 00:09:18 on dist-test-slave-z8x9"
server_key: "ee2f384d310a1884f2bf2eed4cf82849"
server_key_iv: "9773d2cbe9ca165137f5013e34ebe378"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:18.031947 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "303fe15ebd81427ea3b91fb527b2eeef"
format_stamp: "Formatted at 2026-08-12 00:09:18 on dist-test-slave-z8x9"
server_key: "ee2f384d310a1884f2bf2eed4cf82849"
server_key_iv: "9773d2cbe9ca165137f5013e34ebe378"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:18.045717 31472 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.004s sys 0.000s
I20260812 00:09:18.060156 31497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.060776 31472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:18.060904 31472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "303fe15ebd81427ea3b91fb527b2eeef"
format_stamp: "Formatted at 2026-08-12 00:09:18 on dist-test-slave-z8x9"
server_key: "ee2f384d310a1884f2bf2eed4cf82849"
server_key_iv: "9773d2cbe9ca165137f5013e34ebe378"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:18.061022 31472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:18.073971 31472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:18.076251 31472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:18.076432 31472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:18.076726 31472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:18.077265 31472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:18.077328 31472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.077371 31472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:18.077397 31472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.086586 31472 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:37341
I20260812 00:09:18.087086 31472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:09:18.091656 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 31472
I20260812 00:09:18.091817 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:09:18.092048 7511 external_mini_cluster.cc:1428] Setting key c40512671b2032aed89504c766d20263
I20260812 00:09:18.097828 31699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:37341 every 8 connection(s)
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:18.115144 31238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:18.098897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:39337 (local address 127.7.85.254:34823)
0812 00:09:18.098944 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:18.098946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:18.099303 (+ 357us) server_negotiation.cc:407] Connection header received
0812 00:09:18.100045 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:18.100047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:18.100062 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:18.100097 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:18.101027 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.101855 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.103397 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.103712 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.103758 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:18.110769 (+ 7011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:18.110773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:18.110773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:18.110787 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:18.112836 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:18.113543 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:18.113545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:18.113546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:18.113616 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:18.113976 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:18.113978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:18.113978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:18.114148 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:18.114195 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:18.114893 (+ 698us) server_negotiation.cc:299] Negotiation successful
0812 00:09:18.115030 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:09:18.115841 31704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:18.098557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34823 (local address 127.7.85.195:39337)
0812 00:09:18.099165 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:18.099218 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:18.099907 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:18.100310 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:18.100315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:18.100592 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:18.100908 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:18.100916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.101982 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.101983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:18.103300 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:18.103303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.103873 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.103874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:18.104020 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:18.108033 (+ 4013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:18.108049 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:18.110451 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:18.112966 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:18.112979 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:18.112982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:18.113435 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:18.113700 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:18.113701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:18.113701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:18.113880 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:18.114249 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:18.114257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:18.114680 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 00:09:18.115319 (+ 639us) client_negotiation.cc:240] Negotiation successful
0812 00:09:18.115540 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":121,"threads_started":1}
I20260812 00:09:18.116390 31701 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34823
I20260812 00:09:18.116518 31701 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:18.116739 31701 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:18.117194 30868 ts_manager.cc:194] Registered new tserver with Master: 303fe15ebd81427ea3b91fb527b2eeef (127.7.85.195:37341)
I20260812 00:09:18.117779 30868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:39337
I20260812 00:09:18.125820 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:09:18.126760 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 31011
I20260812 00:09:18.194802 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 31241
I20260812 00:09:18.257962 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 31472
I20260812 00:09:18.271750 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 30802
I20260812 00:09:18.315827 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:34823
--webserver_interface=127.7.85.254
--webserver_port=37881
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:34823
--ranger_config_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:18.407397 31705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:18.407631 31705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:18.407655 31705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:18.409126 31705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:18.409243 31705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:18.409269 31705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:18.409314 31705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:18.409334 31705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:18.411216 31705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:34823
--ranger_config_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:34823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=37881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:18.411528 31705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:18.411787 31705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:18.417780 31713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:18.417891 31705 server_base.cc:1034] running on GCE node
W20260812 00:09:18.418049 31716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:18.418198 31714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:18.418514 31705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:18.418859 31705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:18.420051 31705 hybrid_clock.cc:648] HybridClock initialized: now 1786493358420036 us; error 29 us; skew 500 ppm
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:18.423379 31705 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:18.424012 31705 webserver.cc:460] Webserver started at http://127.7.85.254:37881/ using document root <none> and password file <none>
I20260812 00:09:18.424329 31705 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:18.424410 31705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:18.496356 31705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:18.497327 31727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.497506 31705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:18.497634 31705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "a4f9ca5d39de4340afda566a1908e087"
format_stamp: "Formatted at 2026-08-12 00:09:16 on dist-test-slave-z8x9"
server_key: "202df43d7c3ba834709181ed613841f8"
server_key_iv: "afd81d8de94f0f0c4dd535497c7d8cbd"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:18.498008 31705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:18.532721 31705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:18.535054 31705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:18.535212 31705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:18.540532 31705 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:34823
I20260812 00:09:18.540628 31830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:34823 every 8 connection(s)
I20260812 00:09:18.541038 31705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:18.542454 31832 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:09:18.543149 31832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Bootstrap starting.
I20260812 00:09:18.543612 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 31705
I20260812 00:09:18.543931 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:39631
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:34823
--webserver_port=45865
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:18.554505 31832 log.cc:824] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:18.557494 31832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:09:18.558145 31832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Bootstrap complete.
I20260812 00:09:18.562165 31832 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:18.562512 31832 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f9ca5d39de4340afda566a1908e087, State: Initialized, Role: FOLLOWER
I20260812 00:09:18.562734 31832 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:18.562809 31832 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:18.562847 31832 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:18.562928 31832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:09:18.564349 31832 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:18.564469 31832 leader_election.cc:302] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4f9ca5d39de4340afda566a1908e087; no voters:
I20260812 00:09:18.564692 31832 leader_election.cc:288] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:09:18.564781 31841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:09:18.565027 31841 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [term 2 LEADER]: Becoming Leader. State: Replica: a4f9ca5d39de4340afda566a1908e087, State: Running, Role: LEADER
I20260812 00:09:18.565088 31832 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:18.565253 31841 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } }
I20260812 00:09:18.565551 31841 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a4f9ca5d39de4340afda566a1908e087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } } }
I20260812 00:09:18.565623 31841 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:18.565742 31841 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4f9ca5d39de4340afda566a1908e087. Latest consensus state: current_term: 2 leader_uuid: "a4f9ca5d39de4340afda566a1908e087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4f9ca5d39de4340afda566a1908e087" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 34823 } } }
I20260812 00:09:18.565788 31841 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:18.566375 31846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:18.567040 31846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:18.567301 31846 catalog_manager.cc:1259] Loaded cluster ID: 4d3c94fea35944a18540cf637bde7384
I20260812 00:09:18.567348 31832 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 00:09:18.567394 31846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:09:18.569006 31846 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:18.569221 31846 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Loaded TSK: 0
I20260812 00:09:18.569463 31846 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:09:18.672071 31838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:18.672492 31838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:18.672608 31838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:18.674566 31838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:18.674888 31838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:18.675026 31838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:09:18.677528 31838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:39631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=45865
--tserver_master_addrs=127.7.85.254:34823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:18.678081 31838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:18.678442 31838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:18.686429 31908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:18.687736 31903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:18.687894 31838 server_base.cc:1034] running on GCE node
W20260812 00:09:18.688239 31902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:18.688578 31838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:18.688920 31838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:18.690110 31838 hybrid_clock.cc:648] HybridClock initialized: now 1786493358690088 us; error 58 us; skew 500 ppm
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:18.693862 31838 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:09:18.694483 31838 webserver.cc:460] Webserver started at http://127.7.85.193:45865/ using document root <none> and password file <none>
I20260812 00:09:18.694782 31838 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:18.694862 31838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:18.716898 31838 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 00:09:18.718415 31921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.718737 31838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:09:18.718849 31838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "3f728f9d4e924aa1bde0f8b11f456eba"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "8e1f3cb530b6adddd717ed652c352a0b"
server_key_iv: "535ef70b56697ca4f3d2590b47dc2f50"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:18.719343 31838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:18.774148 31838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:18.778522 31838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:18.778726 31838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:18.779021 31838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:18.779472 31838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:18.779534 31838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.779606 31838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:18.779652 31838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.791182 31838 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:39631
I20260812 00:09:18.792543 31838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:09:18.797875 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 31838
I20260812 00:09:18.798207 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:43873
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:34823
--webserver_port=37699
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:18.797494 32101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:39631 every 8 connection(s)
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:18.814465 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:18.798564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34823 (local address 127.7.85.193:36517)
0812 00:09:18.798804 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:18.798864 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:18.799650 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:18.801145 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:18.801150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:18.801470 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:18.801823 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:18.801830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.803936 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.803938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:18.805446 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:18.805452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.806443 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.806445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:18.806603 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:18.807405 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:18.807417 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:18.809344 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:18.812067 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:18.812076 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:18.812079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:18.812413 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:18.812761 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:18.812762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:18.812763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:18.812911 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:18.813454 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:18.813462 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:18.813645 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 00:09:18.814011 (+ 366us) client_negotiation.cc:240] Negotiation successful
0812 00:09:18.814203 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":126,"threads_started":1}
I20260812 00:09:18.814481 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:18.799029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:36517 (local address 127.7.85.254:34823)
0812 00:09:18.800496 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:18.800507 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:18.800556 (+ 49us) server_negotiation.cc:407] Connection header received
0812 00:09:18.800723 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:18.800730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:18.800805 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:18.800957 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:18.801965 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.803444 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.805570 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.806304 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.806436 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:18.809489 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:18.809496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:18.809496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:18.809513 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:18.811952 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:18.812525 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:18.812530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:18.812530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:18.812674 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:18.812997 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:18.812999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:18.812999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:18.813368 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:18.813459 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:18.814118 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 00:09:18.814265 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1355,"thread_start_us":99,"threads_started":1}
I20260812 00:09:18.815940 32102 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34823
I20260812 00:09:18.816051 32102 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:18.816259 32102 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
W20260812 00:09:18.817333 32102 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:34823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 00:09:18.817426 32102 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:18.817507 32102 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:18.817884 32102 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:18.817971 32102 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
W20260812 00:09:18.818352 32102 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:09:18.916380 32111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:18.916662 32111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:18.916699 32111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:18.918460 32111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:18.918529 32111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:18.918570 32111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:09:18.920739 32111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:43873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=37699
--tserver_master_addrs=127.7.85.254:34823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:18.921001 32111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:18.921288 32111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:18.928535 32117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:18.928649 32120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:18.928536 32116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:18.930078 32111 server_base.cc:1034] running on GCE node
I20260812 00:09:18.930352 32111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:18.930648 32111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:18.931828 32111 hybrid_clock.cc:648] HybridClock initialized: now 1786493358931814 us; error 30 us; skew 500 ppm
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:18.934895 32111 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:09:18.935408 32111 webserver.cc:460] Webserver started at http://127.7.85.194:37699/ using document root <none> and password file <none>
I20260812 00:09:18.935708 32111 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:18.935806 32111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:18.956279 32111 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260812 00:09:18.957060 32137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.957352 32111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:18.957469 32111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "cc9071f89f4a4c78995fcab60d41ebb3"
format_stamp: "Formatted at 2026-08-12 00:09:17 on dist-test-slave-z8x9"
server_key: "e9059d13402017ad08fa69b820769960"
server_key_iv: "b10ac97dda2d35810665b219ba4991f1"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:18.957971 32111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:18.968909 32111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:18.971305 32111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:18.971506 32111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:18.971784 32111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:18.972216 32111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:18.972270 32111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.972301 32111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:18.972319 32111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:18.980165 32111 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:43873
I20260812 00:09:18.980286 32320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:43873 every 8 connection(s)
I20260812 00:09:18.980649 32111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:09:18.989710 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 32111
I20260812 00:09:18.990049 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.85.212:54643/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:37341
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:34823
--webserver_port=46861
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:43645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:09:18 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:18.996703 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:18.981864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:55675 (local address 127.7.85.254:34823)
0812 00:09:18.981915 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:18.981917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:18.984589 (+ 2672us) server_negotiation.cc:407] Connection header received
0812 00:09:18.985083 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:18.985085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:18.985103 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:18.985218 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:18.986173 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.986984 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.989324 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.989636 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.989717 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:18.992789 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:18.992790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:18.992791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:18.992801 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:18.994791 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:18.995401 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:18.995403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:18.995403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:18.995476 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:18.995836 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:18.995837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:18.995837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:18.996003 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:18.996090 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:18.996493 (+ 403us) server_negotiation.cc:299] Negotiation successful
0812 00:09:18.996598 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:09:18.997408 32324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:18.981551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34823 (local address 127.7.85.194:55675)
0812 00:09:18.984411 (+ 2860us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:18.984458 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:18.984965 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:18.985626 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:18.985653 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:18.985858 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:18.986066 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:18.986070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.987217 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.987219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:18.989040 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:18.989047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:18.989882 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:18.989885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:18.990063 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:18.990733 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:18.990745 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:18.992647 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:18.994902 (+ 2255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:18.994911 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:18.994915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:18.995319 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:18.995567 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:18.995568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:18.995568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:18.995747 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:18.996094 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:18.996101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:18.996309 (+ 208us) client_negotiation.cc:769] Sending connection context
0812 00:09:18.996445 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 00:09:18.996653 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2811,"thread_start_us":86,"threads_started":1}
I20260812 00:09:18.998064 32322 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34823
I20260812 00:09:18.998196 32322 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:18.998432 32322 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
W20260812 00:09:18.999229 32322 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:34823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 00:09:18.999337 32322 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:18.999408 32322 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:19.000164 32322 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:19.000252 32322 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
W20260812 00:09:19.000635 32322 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:09:19.095732 32325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:19.096025 32325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:19.096068 32325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:19.097458 32325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:19.097507 32325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:19.097538 32325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:09:19.099234 32325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:43645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.85.212:54643/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:37341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=46861
--tserver_master_addrs=127.7.85.254:34823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:19.099526 32325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:19.099885 32325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:19.106396 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:19.106599 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:19.107730 32338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:19.108070 32325 server_base.cc:1034] running on GCE node
I20260812 00:09:19.108347 32325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:19.108696 32325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:19.109911 32325 hybrid_clock.cc:648] HybridClock initialized: now 1786493359109879 us; error 48 us; skew 500 ppm
Aug 12 00:09:19 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:19.113476 32325 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:09:19.114065 32325 webserver.cc:460] Webserver started at http://127.7.85.195:46861/ using document root <none> and password file <none>
I20260812 00:09:19.114379 32325 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:19.114459 32325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:09:19 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:19.141726 32325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:09:19.148164 32352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:19.148365 32325 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260812 00:09:19.148458 32325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "303fe15ebd81427ea3b91fb527b2eeef"
format_stamp: "Formatted at 2026-08-12 00:09:18 on dist-test-slave-z8x9"
server_key: "ee2f384d310a1884f2bf2eed4cf82849"
server_key_iv: "9773d2cbe9ca165137f5013e34ebe378"
server_key_version: "kuduclusterkey@0"
I20260812 00:09:19.148943 32325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:19.172362 32325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:19.175057 32325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:19.175257 32325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:19.175529 32325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:19.175935 32325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:19.176007 32325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:19.176051 32325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:19.176077 32325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:19.184667 32325 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:37341
I20260812 00:09:19.184885 32559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:37341 every 8 connection(s)
I20260812 00:09:19.185376 32325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:09:19.188417 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 32325
Aug 12 00:09:19 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:19.197772 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:19.186646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:47287 (local address 127.7.85.254:34823)
0812 00:09:19.186720 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:19.186723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:19.186800 (+ 77us) server_negotiation.cc:407] Connection header received
0812 00:09:19.187480 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:19.187481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:19.187491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:19.187540 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:19.188249 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:19.188922 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:19.190516 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:19.190824 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:19.190867 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:19.193857 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:19.193858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:19.193859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:19.193868 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:19.195874 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:19.196462 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:19.196463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:19.196464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:19.196534 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:19.196869 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:19.196870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:19.196870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:19.197016 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:19.197081 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:19.197562 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 00:09:19.197660 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 00:09:19.198392 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:19.186246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:34823 (local address 127.7.85.195:47287)
0812 00:09:19.186643 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:19.186694 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:19.187368 (+ 674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:19.187616 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:19.187620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:19.187860 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:19.188127 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:19.188132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:19.189047 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:19.189049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:19.190406 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:19.190409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:19.190978 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:19.190980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:19.191120 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:19.191738 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:19.191749 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:19.193765 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:19.195987 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:19.195996 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:19.196000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:19.196363 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:19.196627 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:19.196628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:19.196628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:19.196784 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:19.197150 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:19.197157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:19.197376 (+ 219us) client_negotiation.cc:769] Sending connection context
0812 00:09:19.197929 (+ 553us) client_negotiation.cc:240] Negotiation successful
0812 00:09:19.198117 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":214,"threads_started":1}
I20260812 00:09:19.198894 32561 heartbeater.cc:347] Connected to a master server at 127.7.85.254:34823
I20260812 00:09:19.199008 32561 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:19.199083 32570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:19.189255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.7.85.254:34823)
0812 00:09:19.189399 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:19.189401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:19.189554 (+ 153us) server_negotiation.cc:407] Connection header received
0812 00:09:19.189632 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:19.189633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:19.189645 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:19.189723 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:19.190340 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:19.190942 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:19.192713 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:19.193047 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:19.193137 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:19.194973 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:19.194974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:19.194975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:19.194982 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:19.197100 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:19.197486 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:19.197488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:19.197488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:19.197554 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:19.198271 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:19.198273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:19.198273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:19.198400 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:19.198710 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:19.198811 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 00:09:19.198900 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260812 00:09:19.199232 32561 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
W20260812 00:09:19.199928 32561 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:34823 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 00:09:19.200018 32561 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:19.200116 32561 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:19.200496 32561 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:19.200580 32561 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
W20260812 00:09:19.200950 32561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 00:09:19 dist-test-slave-z8x9 krb5kdc[29979](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786493359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:19.818554 32102 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:19.818768 32102 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:19.819209 31757 ts_manager.cc:194] Registered new tserver with Master: 3f728f9d4e924aa1bde0f8b11f456eba (127.7.85.193:39631)
I20260812 00:09:19.820384 31757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:36517
I20260812 00:09:20.000857 32322 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:20.001055 32322 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:20.001667 31757 ts_manager.cc:194] Registered new tserver with Master: cc9071f89f4a4c78995fcab60d41ebb3 (127.7.85.194:43873)
I20260812 00:09:20.002130 31757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:55675
Aug 12 00:09:20 dist-test-slave-z8x9 krb5kdc[29979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for HTTP/127.7.85.212@KRBTEST.COM
I20260812 00:09:20.201184 32561 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:20.201359 32561 heartbeater.cc:511] Master 127.7.85.254:34823 requested a full tablet report, sending...
I20260812 00:09:20.201741 31757 ts_manager.cc:194] Registered new tserver with Master: 303fe15ebd81427ea3b91fb527b2eeef (127.7.85.195:37341)
I20260812 00:09:20.202314 31757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:47287
I20260812 00:09:20.212496 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:09:20.225399 32600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.216282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.7.85.254:34823)
0812 00:09:20.216606 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.216608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.216730 (+ 122us) server_negotiation.cc:407] Connection header received
0812 00:09:20.216835 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.216836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.216849 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.216894 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:20.217734 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.218558 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.219645 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.220057 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.220121 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.221963 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:20.221965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:20.221965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:20.221974 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:20.223936 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:20.224429 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:20.224430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:20.224430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:20.224499 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:20.224699 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:20.224700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:20.224700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:20.224848 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:20.224927 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.225157 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.225277 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":95,"threads_started":1}
I20260812 00:09:20.228107 31757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:09:20.230509 31757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:09:20.245229 32609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.234645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:37341 (local address 127.0.0.1:51466)
0812 00:09:20.235798 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:20.235806 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:20.235840 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:20.236424 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:20.236428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:20.236432 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:20.236513 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.236515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.241930 (+ 5415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.241931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.243417 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.243421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.244803 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.244804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.244924 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.244941 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 00:09:20.245017 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:09:20.245080 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"thread_start_us":82,"threads_started":1}
I20260812 00:09:20.247227 32613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.234429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:43873 (local address 127.0.0.1:45916)
0812 00:09:20.235566 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:20.235616 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:20.235683 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:20.237796 (+ 2113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:20.237798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:20.237802 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:20.237852 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.237853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.239113 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.239114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.240877 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.240881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.246933 (+ 6052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.246934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.247018 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.247023 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:09:20.247075 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 00:09:20.247126 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1097,"thread_start_us":84,"threads_started":1}
I20260812 00:09:20.250548 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.234995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.7.85.195:37341)
0812 00:09:20.236044 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.236050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.236060 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:09:20.236097 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.236099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.236115 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.236187 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:20.236612 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.241824 (+ 5212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.243526 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.244593 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.244679 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.244757 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.250396 (+ 5639us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.250461 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":59,"threads_started":1}
I20260812 00:09:20.249920 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.234526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.7.85.194:43873)
0812 00:09:20.237549 (+ 3023us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.237554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.237565 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:09:20.237601 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.237603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.237619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.237695 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:20.237927 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.239010 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.245492 (+ 6482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.246745 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.246825 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.246889 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.249768 (+ 2879us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.249825 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2982,"thread_start_us":75,"threads_started":1}
I20260812 00:09:20.251508 32412 tablet_service.cc:1467] Processing CreateTablet for tablet 8f345892348945cdb396e61614038643 (DEFAULT_TABLE table=test-table [id=8b25447e1cd64612b7a0ecbaf9591423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:20.251968 32412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f345892348945cdb396e61614038643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:20.255023 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.234855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:39631 (local address 127.0.0.1:52330)
0812 00:09:20.235902 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:20.235910 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:20.235941 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:20.247740 (+ 11799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:20.247742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:20.247745 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:20.247791 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.247793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.251180 (+ 3387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.251434 (+ 254us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.253099 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.253104 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.254745 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.254747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.254826 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.254834 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:09:20.254867 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 00:09:20.254934 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"thread_start_us":71,"threads_started":1}
I20260812 00:09:20.256191 32616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.234954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.7.85.193:39631)
0812 00:09:20.247472 (+ 12518us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.247479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.247491 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:09:20.247532 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.247534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.247555 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.247637 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:20.247866 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.249594 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.253284 (+ 3690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.254618 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.255579 (+ 961us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.255892 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.255960 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.256036 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12473,"thread_start_us":68,"threads_started":1}
I20260812 00:09:20.256542 32219 tablet_service.cc:1467] Processing CreateTablet for tablet 8f345892348945cdb396e61614038643 (DEFAULT_TABLE table=test-table [id=8b25447e1cd64612b7a0ecbaf9591423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:20.256942 32219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f345892348945cdb396e61614038643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:20.257274 31981 tablet_service.cc:1467] Processing CreateTablet for tablet 8f345892348945cdb396e61614038643 (DEFAULT_TABLE table=test-table [id=8b25447e1cd64612b7a0ecbaf9591423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:20.257625 31981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f345892348945cdb396e61614038643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:20.266903 32619 tablet_bootstrap.cc:492] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Bootstrap starting.
I20260812 00:09:20.267777 32619 tablet_bootstrap.cc:654] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:20.268126 32619 log.cc:824] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:20.268560 32618 tablet_bootstrap.cc:492] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Bootstrap starting.
I20260812 00:09:20.269446 32618 tablet_bootstrap.cc:654] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:20.270143 32618 log.cc:824] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:20.270592 32620 tablet_bootstrap.cc:492] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: Bootstrap starting.
I20260812 00:09:20.271881 32620 tablet_bootstrap.cc:654] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:20.272219 32620 log.cc:824] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:20.273381 32619 tablet_bootstrap.cc:492] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: No bootstrap required, opened a new log
I20260812 00:09:20.273473 32619 ts_tablet_manager.cc:1397] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 00:09:20.279126 32619 raft_consensus.cc:348] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.279367 32619 raft_consensus.cc:374] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:20.279402 32619 raft_consensus.cc:729] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9071f89f4a4c78995fcab60d41ebb3, State: Initialized, Role: FOLLOWER
I20260812 00:09:20.279541 32619 consensus_queue.cc:260] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.279871 32619 ts_tablet_manager.cc:1428] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260812 00:09:20.280007 32322 heartbeater.cc:503] Master 127.7.85.254:34823 was elected leader, sending a full tablet report...
I20260812 00:09:20.281029 32618 tablet_bootstrap.cc:492] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: No bootstrap required, opened a new log
I20260812 00:09:20.281121 32618 ts_tablet_manager.cc:1397] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20260812 00:09:20.283721 32618 raft_consensus.cc:348] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.283923 32618 raft_consensus.cc:374] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:20.283955 32618 raft_consensus.cc:729] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 303fe15ebd81427ea3b91fb527b2eeef, State: Initialized, Role: FOLLOWER
I20260812 00:09:20.284112 32618 consensus_queue.cc:260] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.284399 32618 ts_tablet_manager.cc:1428] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 00:09:20.284524 32561 heartbeater.cc:503] Master 127.7.85.254:34823 was elected leader, sending a full tablet report...
I20260812 00:09:20.286691 32620 tablet_bootstrap.cc:492] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: No bootstrap required, opened a new log
I20260812 00:09:20.286799 32620 ts_tablet_manager.cc:1397] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260812 00:09:20.289999 32620 raft_consensus.cc:348] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.290215 32620 raft_consensus.cc:374] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:20.290253 32620 raft_consensus.cc:729] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f728f9d4e924aa1bde0f8b11f456eba, State: Initialized, Role: FOLLOWER
I20260812 00:09:20.290412 32620 consensus_queue.cc:260] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.290736 32620 ts_tablet_manager.cc:1428] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 00:09:20.290776 32102 heartbeater.cc:503] Master 127.7.85.254:34823 was elected leader, sending a full tablet report...
W20260812 00:09:20.299350 32103 tablet.cc:2369] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:09:20.383096 32628 raft_consensus.cc:482] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:20.383271 32628 raft_consensus.cc:504] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.383674 32628 leader_election.cc:288] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc9071f89f4a4c78995fcab60d41ebb3 (127.7.85.194:43873), 303fe15ebd81427ea3b91fb527b2eeef (127.7.85.195:37341)
I20260812 00:09:20.387977 32610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.383976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:58577 (local address 127.7.85.194:43873)
0812 00:09:20.384026 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.384028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.384225 (+ 197us) server_negotiation.cc:407] Connection header received
0812 00:09:20.384332 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.384334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.384347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.384375 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:20.384649 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.385501 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.387035 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.387560 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.387609 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.387618 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.387856 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.387899 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:09:20.388047 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.384358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:33941 (local address 127.7.85.195:37341)
0812 00:09:20.384441 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.384443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.384528 (+ 85us) server_negotiation.cc:407] Connection header received
0812 00:09:20.384641 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.384642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.384654 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.384699 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:20.384936 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.385686 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.387044 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.387607 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.387676 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.387685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.387926 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.387972 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:09:20.388329 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.384138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:37341 (local address 127.7.85.193:33941)
0812 00:09:20.384457 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:20.384468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:20.384559 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:20.384774 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:20.384775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:20.384777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:20.384851 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.384853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.385792 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.385793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.386938 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.386942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.387753 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.387753 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.387833 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.387837 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:20.388178 (+ 341us) client_negotiation.cc:240] Negotiation successful
0812 00:09:20.388229 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":91,"threads_started":1}
I20260812 00:09:20.388614 32629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.383888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:43873 (local address 127.7.85.193:58577)
0812 00:09:20.384147 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:20.384159 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:20.384257 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:20.384492 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:20.384493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:20.384496 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:20.384580 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.384581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.385612 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.385613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.386928 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:20.386931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.387700 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.387701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:20.387779 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.387783 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:20.388481 (+ 698us) client_negotiation.cc:240] Negotiation successful
0812 00:09:20.388532 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":60,"thread_start_us":125,"threads_started":1}
I20260812 00:09:20.388736 32446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f345892348945cdb396e61614038643" candidate_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303fe15ebd81427ea3b91fb527b2eeef" is_pre_election: true
I20260812 00:09:20.388934 32446 raft_consensus.cc:2454] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f728f9d4e924aa1bde0f8b11f456eba in term 0.
I20260812 00:09:20.389217 32245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f345892348945cdb396e61614038643" candidate_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" is_pre_election: true
I20260812 00:09:20.389382 32245 raft_consensus.cc:2454] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f728f9d4e924aa1bde0f8b11f456eba in term 0.
I20260812 00:09:20.389389 31925 leader_election.cc:302] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 303fe15ebd81427ea3b91fb527b2eeef, 3f728f9d4e924aa1bde0f8b11f456eba; no voters:
I20260812 00:09:20.389526 32628 raft_consensus.cc:2781] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:09:20.389593 32628 raft_consensus.cc:482] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:09:20.389628 32628 raft_consensus.cc:3037] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:20.390583 32628 raft_consensus.cc:504] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.390743 32628 leader_election.cc:288] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [CANDIDATE]: Term 1 election: Requested vote from peers cc9071f89f4a4c78995fcab60d41ebb3 (127.7.85.194:43873), 303fe15ebd81427ea3b91fb527b2eeef (127.7.85.195:37341)
I20260812 00:09:20.390975 32245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f345892348945cdb396e61614038643" candidate_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc9071f89f4a4c78995fcab60d41ebb3"
I20260812 00:09:20.390981 32446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f345892348945cdb396e61614038643" candidate_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "303fe15ebd81427ea3b91fb527b2eeef"
I20260812 00:09:20.391057 32245 raft_consensus.cc:3037] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:20.391067 32446 raft_consensus.cc:3037] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:20.392059 32446 raft_consensus.cc:2454] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f728f9d4e924aa1bde0f8b11f456eba in term 1.
I20260812 00:09:20.392181 32245 raft_consensus.cc:2454] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f728f9d4e924aa1bde0f8b11f456eba in term 1.
I20260812 00:09:20.392336 31925 leader_election.cc:302] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f728f9d4e924aa1bde0f8b11f456eba, cc9071f89f4a4c78995fcab60d41ebb3; no voters:
I20260812 00:09:20.392462 32628 raft_consensus.cc:2781] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:20.392617 32628 raft_consensus.cc:686] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [term 1 LEADER]: Becoming Leader. State: Replica: 3f728f9d4e924aa1bde0f8b11f456eba, State: Running, Role: LEADER
I20260812 00:09:20.392764 32628 consensus_queue.cc:237] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } }
I20260812 00:09:20.393621 31757 catalog_manager.cc:5495] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f728f9d4e924aa1bde0f8b11f456eba (127.7.85.193). New cstate: current_term: 1 leader_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f728f9d4e924aa1bde0f8b11f456eba" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 39631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 } health_report { overall_health: UNKNOWN } } }
I20260812 00:09:20.423978 32616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.420288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.7.85.193:39631)
0812 00:09:20.420359 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:20.420362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:20.420371 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:09:20.420416 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:20.420418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:20.420442 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:20.420528 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:09:20.420718 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.421519 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.422725 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:20.423055 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:20.423149 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:20.423416 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:09:20.423565 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:09:20.423653 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:20.423862 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 00:09:20.423885 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:09:20.429026 32446 raft_consensus.cc:1261] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 1 FOLLOWER]: Refusing update from remote peer 3f728f9d4e924aa1bde0f8b11f456eba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:20.429042 32245 raft_consensus.cc:1261] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 1 FOLLOWER]: Refusing update from remote peer 3f728f9d4e924aa1bde0f8b11f456eba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:20.429406 32628 consensus_queue.cc:1035] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [LEADER]: Connected to new peer: Peer: permanent_uuid: "303fe15ebd81427ea3b91fb527b2eeef" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 37341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:20.429571 32633 consensus_queue.cc:1035] T 8f345892348945cdb396e61614038643 P 3f728f9d4e924aa1bde0f8b11f456eba [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc9071f89f4a4c78995fcab60d41ebb3" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:20.432950 32636 mvcc.cc:205] Tried to move back new op lower bound from 7317476804313939968 to 7317476804169433088. Current Snapshot: MvccSnapshot[applied={T|T < 7317476804313939968}]
I20260812 00:09:20.434329 32639 mvcc.cc:205] Tried to move back new op lower bound from 7317476804313939968 to 7317476804169433088. Current Snapshot: MvccSnapshot[applied={T|T < 7317476804313939968}]
I20260812 00:09:20.435189 32641 mvcc.cc:205] Tried to move back new op lower bound from 7317476804313939968 to 7317476804169433088. Current Snapshot: MvccSnapshot[applied={T|T < 7317476804313939968}]
I20260812 00:09:20.437242 31756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:20.437358 31756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:20.438295 31756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4f9ca5d39de4340afda566a1908e087: Sending DeleteTablet for 3 replicas of tablet 8f345892348945cdb396e61614038643
I20260812 00:09:20.438861 32219 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f345892348945cdb396e61614038643 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:20 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45916
I20260812 00:09:20.439131 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 31838
I20260812 00:09:20.439121 32412 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f345892348945cdb396e61614038643 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:20 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:51466
I20260812 00:09:20.450789 32652 tablet_replica.cc:330] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: stopping tablet replica
I20260812 00:09:20.450991 32652 raft_consensus.cc:2229] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:20.451128 32652 raft_consensus.cc:2258] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:20.453665 32652 ts_tablet_manager.cc:1905] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:20.454258 32651 tablet_replica.cc:330] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: stopping tablet replica
I20260812 00:09:20.454388 32651 raft_consensus.cc:2229] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:20.454507 32651 raft_consensus.cc:2258] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:20.456643 32651 ts_tablet_manager.cc:1905] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:20.458271 32652 ts_tablet_manager.cc:1918] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:20.458377 32652 log.cc:1196] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/8f345892348945cdb396e61614038643
I20260812 00:09:20.458694 32652 ts_tablet_manager.cc:1939] T 8f345892348945cdb396e61614038643 P 303fe15ebd81427ea3b91fb527b2eeef: Deleting consensus metadata
I20260812 00:09:20.459234 31735 catalog_manager.cc:4841] TS 303fe15ebd81427ea3b91fb527b2eeef (127.7.85.195:37341): tablet 8f345892348945cdb396e61614038643 (table test-table [id=8b25447e1cd64612b7a0ecbaf9591423]) successfully deleted
I20260812 00:09:20.459738 32651 ts_tablet_manager.cc:1918] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:20.459834 32651 log.cc:1196] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/8f345892348945cdb396e61614038643
I20260812 00:09:20.460115 32651 ts_tablet_manager.cc:1939] T 8f345892348945cdb396e61614038643 P cc9071f89f4a4c78995fcab60d41ebb3: Deleting consensus metadata
I20260812 00:09:20.460673 31735 catalog_manager.cc:4841] TS cc9071f89f4a4c78995fcab60d41ebb3 (127.7.85.194:43873): tablet 8f345892348945cdb396e61614038643 (table test-table [id=8b25447e1cd64612b7a0ecbaf9591423]) successfully deleted
W20260812 00:09:20.461313 31732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:39631 (error 108)
I20260812 00:09:20.461706 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 32111
I20260812 00:09:20.476146 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:20.475965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:39631 (local address 127.0.0.1:52352)
0812 00:09:20.476038 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:20.476094 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:39631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 00:09:20.476325 31732 catalog_manager.cc:4584] TS 3f728f9d4e924aa1bde0f8b11f456eba (127.7.85.193:39631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f345892348945cdb396e61614038643: Network error: Client connection negotiation failed: client connection to 127.7.85.193:39631: connect: Connection refused (error 111)
I20260812 00:09:20.488494 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 32325
I20260812 00:09:20.503114 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 31705
I20260812 00:09:20.524447 7511 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 00:09:20.882587 7511 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 00:09:20.882714 7511 mini_ranger.cc:66] Stopping Ranger...
I20260812 00:09:21.239554 7511 mini_ranger.cc:68] Stopped Ranger
2026-08-12 00:09:21.239 UTC [30036] LOG: received smart shutdown request
2026-08-12 00:09:21.245 UTC [30036] LOG: background worker "logical replication launcher" (PID 30053) exited with exit code 1
2026-08-12 00:09:21.245 UTC [30044] LOG: shutting down
2026-08-12 00:09:21.489 UTC [30036] LOG: database system is shut down
2026-08-12T00:09:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54448 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:09:21 dist-test-slave-z8x9 krb5kdc[32674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:09:21 dist-test-slave-z8x9 krb5kdc[32674](info): set up 2 sockets
Aug 12 00:09:21 dist-test-slave-z8x9 krb5kdc[32674](info): commencing operation
krb5kdc: starting...
W20260812 00:09:23.012809 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:09:23.158190 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41297
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41297,127.7.85.253:38325,127.7.85.252:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:23.252840 32699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:23.253155 32699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:23.253230 32699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:23.254519 32699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:23.254609 32699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:23.254632 32699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:23.254658 32699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:23.254688 32699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:23.256389 32699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41297,127.7.85.253:38325,127.7.85.252:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:23.256735 32699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:23.257012 32699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:23.263231 32707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:23.263350 32711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:23.263660 32699 server_base.cc:1034] running on GCE node
W20260812 00:09:23.263257 32709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:23.264648 32699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:23.265273 32699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:23.266736 32699 hybrid_clock.cc:648] HybridClock initialized: now 1786493363266697 us; error 50 us; skew 500 ppm
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:23.270761 32699 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:23.271339 32699 webserver.cc:460] Webserver started at http://127.7.85.254:43999/ using document root <none> and password file <none>
I20260812 00:09:23.271615 32699 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:23.271675 32699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:23.271831 32699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:23.273017 32699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "3f76e8bd69ee4cd18b5fd434396a86ad"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "b9c839d046e52c2d05ac67ec0e28c122"
server_key_iv: "8ff1f3378e4e893fb179948e602a3d0a"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.273473 32699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "3f76e8bd69ee4cd18b5fd434396a86ad"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "b9c839d046e52c2d05ac67ec0e28c122"
server_key_iv: "8ff1f3378e4e893fb179948e602a3d0a"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.275056 32699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:23.276201 32720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:23.276952 32699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:23.277314 32699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "3f76e8bd69ee4cd18b5fd434396a86ad"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "b9c839d046e52c2d05ac67ec0e28c122"
server_key_iv: "8ff1f3378e4e893fb179948e602a3d0a"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.277626 32699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:23.321914 32699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:23.324455 32699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:23.324661 32699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:23.329829 32699 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:41297
I20260812 00:09:23.329864 331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:41297 every 8 connection(s)
I20260812 00:09:23.330297 32699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:23.331334 332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:23.333077 332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.337030 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 32699
I20260812 00:09:23.337236 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:09:23.337486 7511 external_mini_cluster.cc:1428] Setting key 93e213fa6ccf06072f864dc62402eb08
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:23.345373 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.333974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41297 (local address 127.0.0.1:44574)
0812 00:09:23.334314 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.334393 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.334972 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.335285 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.335288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.335585 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.335749 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.335754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.337226 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.337227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.338045 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.338048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.338598 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.338599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.338673 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.339364 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.339377 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.340872 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.343558 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.343567 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.343569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.343808 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.344135 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.344136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.344136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.344227 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.344736 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.344742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.344805 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.344900 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.345046 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":174,"threads_started":1}
I20260812 00:09:23.345386 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.334271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.7.85.254:41297)
0812 00:09:23.334401 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.334405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.334515 (+ 110us) server_negotiation.cc:407] Connection header received
0812 00:09:23.335147 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.335149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.335168 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.335272 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.335862 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.336949 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.338152 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.338475 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.338632 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.340972 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.340981 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.340981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.340991 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.343445 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.343926 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.343931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.343932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.344056 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.344298 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.344299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.344299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.344660 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.344759 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.344975 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.345136 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":101,"thread_start_us":47,"threads_started":1}
I20260812 00:09:23.346378 332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.346793 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.346663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60460)
0812 00:09:23.346717 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.346771 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 00:09:23.346935 32721 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.85.253@KRBTEST.COM; defaulting to no policy
I20260812 00:09:23.347407 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.347304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60466)
0812 00:09:23.347354 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.347388 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 00:09:23.347509 332 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:38325: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab.
Entry for principal kudu/127.7.85.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab.
I20260812 00:09:23.370281 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:38325
--webserver_interface=127.7.85.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab
--principal=kudu/127.7.85.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41297,127.7.85.253:38325,127.7.85.252:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:23.396727 332 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } attempt: 1
I20260812 00:09:23.397676 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.397114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60470)
0812 00:09:23.397247 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.397428 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 00:09:23.398097 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.397971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60476)
0812 00:09:23.398012 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.398050 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 00:09:23.398211 332 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:38325: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
W20260812 00:09:23.464489 352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:23.464711 352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:23.464735 352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:23.465864 352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:23.465903 352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:23.465917 352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:23.465929 352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:23.465942 352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:23.466382 332 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } attempt: 2
I20260812 00:09:23.466914 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.466756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60482)
0812 00:09:23.466840 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.466879 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 00:09:23.467423 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.467285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60492)
0812 00:09:23.467356 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.467399 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 00:09:23.467572 332 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:38325: Network error: Client connection negotiation failed: client connection to 127.7.85.253:38325: connect: Connection refused (error 111)
I20260812 00:09:23.467913 352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41297,127.7.85.253:38325,127.7.85.252:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab
--principal=kudu/127.7.85.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:38325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.85.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:23.468151 352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:23.468339 352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:23.473718 366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:23.473718 359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:23.473735 360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:23.474053 352 server_base.cc:1034] running on GCE node
I20260812 00:09:23.474345 352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:23.474707 352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:23.475901 352 hybrid_clock.cc:648] HybridClock initialized: now 1786493363475862 us; error 58 us; skew 500 ppm
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:23.478921 352 init.cc:377] Logged in from keytab as kudu/127.7.85.253@KRBTEST.COM (short username kudu)
I20260812 00:09:23.479451 352 webserver.cc:460] Webserver started at http://127.7.85.253:35695/ using document root <none> and password file <none>
I20260812 00:09:23.479837 352 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:23.480055 352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:23.480420 352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:23.482079 352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data/instance:
uuid: "914084140d5b4fb69fcd293d141ff34b"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "e4ea2292a0d2180b46d6cfd8330534d1"
server_key_iv: "bc55604496db56285ed5ced387f47f7d"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.482470 352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal/instance:
uuid: "914084140d5b4fb69fcd293d141ff34b"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "e4ea2292a0d2180b46d6cfd8330534d1"
server_key_iv: "bc55604496db56285ed5ced387f47f7d"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.483992 352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:23.484917 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:23.485255 352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:23.485374 352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal
uuid: "914084140d5b4fb69fcd293d141ff34b"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "e4ea2292a0d2180b46d6cfd8330534d1"
server_key_iv: "bc55604496db56285ed5ced387f47f7d"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.485432 352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:23.499828 352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:23.502626 352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:23.502771 352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:23.507486 352 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.253:38325
I20260812 00:09:23.507573 475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.253:38325 every 8 connection(s)
I20260812 00:09:23.507934 352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
I20260812 00:09:23.508962 476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:23.510694 476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.517079 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 352
I20260812 00:09:23.517256 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-1/wal/instance
I20260812 00:09:23.517443 7511 external_mini_cluster.cc:1428] Setting key cec008b88af832216cfce5f2192f1efb
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:23.523249 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.511560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.7.85.254:41297)
0812 00:09:23.511632 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.511634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.512186 (+ 552us) server_negotiation.cc:407] Connection header received
0812 00:09:23.513063 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.513064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.513077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.513144 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.513978 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.514496 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.515797 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.516019 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.516081 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.519840 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.519842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.519842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.519850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.521571 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.522090 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.522092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.522092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.522158 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.522521 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.522522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.522522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.522657 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.522738 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.523051 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.523148 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:09:23.523480 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.511465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41297 (local address 127.0.0.1:44580)
0812 00:09:23.511942 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.512020 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.512919 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.513336 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.513340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.513648 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.513878 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.513883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.514622 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.514623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.515691 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.515695 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.516115 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.516116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.516248 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.516925 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.516941 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.519676 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.521686 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.521698 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.521702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.521993 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.522319 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.522320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.522320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.522438 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.522729 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.522738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.522899 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.523015 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.523212 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":263,"threads_started":1}
I20260812 00:09:23.524191 476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.85.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.252@KRBTEST.COM" created.
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:09:23.536926 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.524517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60496)
0812 00:09:23.524573 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.524586 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.524636 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.524942 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.524943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.525106 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.525159 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.525161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.526096 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.526097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.526684 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.526686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.527177 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.527178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.527248 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.528012 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.528017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.532155 (+ 4138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.535615 (+ 3460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.535616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.535617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.535878 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.536128 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.536129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.536129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.536190 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.536511 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.536512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.536555 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.536641 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.536799 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260812 00:09:23.537000 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.524597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.7.85.253:38325)
0812 00:09:23.524803 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.524807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.524825 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:09:23.524843 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.524845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.524857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.524947 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.525271 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.525991 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.526780 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.527081 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.527181 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.532348 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.532370 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.532370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.532381 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.535469 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.535980 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.535991 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.535992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.536051 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.536257 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.536258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.536258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.536428 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.536524 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.536678 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.536865 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":174,"thread_start_us":112,"threads_started":1}
I20260812 00:09:23.538064 476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.538525 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.538383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49782)
0812 00:09:23.538439 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.538491 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 00:09:23.538676 381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
I20260812 00:09:23.539124 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.539023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49796)
0812 00:09:23.539071 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.539097 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 00:09:23.539237 476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:42255: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab.
Entry for principal kudu/127.7.85.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab.
I20260812 00:09:23.553931 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:42255
--webserver_interface=127.7.85.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab
--principal=kudu/127.7.85.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:46699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41297,127.7.85.253:38325,127.7.85.252:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:23.558792 332 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } attempt: 3
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:09:23.567607 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.559219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60498)
0812 00:09:23.559292 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.559306 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.559379 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.559553 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.559554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.559735 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.559806 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.559808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.560611 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.560613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.561224 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.561227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.561868 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.561870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.561961 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.562761 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.562767 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.564432 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.566492 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.566494 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.566494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.566685 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.566911 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.566912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.566912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.566980 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.567244 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.567245 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.567298 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.567386 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.567500 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 00:09:23.567607 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.559298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.7.85.253:38325)
0812 00:09:23.559369 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.559371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.559383 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:09:23.559450 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.559452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.559464 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.559547 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.559894 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.560423 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.561327 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.561745 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.561869 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.564533 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.564536 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.564536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.564546 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.566372 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.566765 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.566766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.566767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.566832 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.567046 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.567047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.567047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.567172 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.567253 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.567414 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.567501 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:09:23.568271 332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.568707 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.568548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49804)
0812 00:09:23.568634 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.568671 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 00:09:23.569087 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.568985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49818)
0812 00:09:23.569036 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.569061 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 00:09:23.569224 332 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:42255: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
I20260812 00:09:23.588500 476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } attempt: 1
I20260812 00:09:23.589113 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.588962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49828)
0812 00:09:23.589035 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.589079 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 00:09:23.589704 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.589456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49832)
0812 00:09:23.589575 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.589650 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 00:09:23.589932 476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:42255: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
I20260812 00:09:23.600430 332 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } attempt: 1
I20260812 00:09:23.601341 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.601034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49834)
0812 00:09:23.601143 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.601258 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 00:09:23.601811 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.601680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49844)
0812 00:09:23.601713 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.601788 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 00:09:23.601934 332 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:42255: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
W20260812 00:09:23.647295 487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:23.647541 487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:23.647567 487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:23.648693 487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:23.648736 487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:23.648751 487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:23.648763 487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:23.648777 487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:23.650452 487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:46699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41297,127.7.85.253:38325,127.7.85.252:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab
--principal=kudu/127.7.85.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.85.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:23.650681 487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:23.650882 487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:23.656905 498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:23.656966 496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:23.657158 495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:23.657198 487 server_base.cc:1034] running on GCE node
I20260812 00:09:23.657670 487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:23.658093 487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:23.658147 476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } attempt: 2
I20260812 00:09:23.658622 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.658479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49858)
0812 00:09:23.658549 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.658581 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:09:23.659049 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.658896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49872)
0812 00:09:23.658985 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.659019 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260812 00:09:23.659188 476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:42255: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
I20260812 00:09:23.659288 487 hybrid_clock.cc:648] HybridClock initialized: now 1786493363659270 us; error 41 us; skew 500 ppm
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:23.662317 487 init.cc:377] Logged in from keytab as kudu/127.7.85.252@KRBTEST.COM (short username kudu)
I20260812 00:09:23.662984 487 webserver.cc:460] Webserver started at http://127.7.85.252:36957/ using document root <none> and password file <none>
I20260812 00:09:23.663300 487 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:23.663388 487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:23.663563 487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:23.664808 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data/instance:
uuid: "09b5266df5ce480f96313f84f74be23f"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "58662af81113d9136210f1f4bf7dba0c"
server_key_iv: "a4093d324c2df8416182aeac691045aa"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.665297 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal/instance:
uuid: "09b5266df5ce480f96313f84f74be23f"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "58662af81113d9136210f1f4bf7dba0c"
server_key_iv: "a4093d324c2df8416182aeac691045aa"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.667006 487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 00:09:23.667965 510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:23.668205 487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:23.668350 487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal
uuid: "09b5266df5ce480f96313f84f74be23f"
format_stamp: "Formatted at 2026-08-12 00:09:23 on dist-test-slave-z8x9"
server_key: "58662af81113d9136210f1f4bf7dba0c"
server_key_iv: "a4093d324c2df8416182aeac691045aa"
server_key_version: "encryptionkey@0"
I20260812 00:09:23.668438 487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:23.677099 332 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } attempt: 2
I20260812 00:09:23.677719 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.677563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49886)
0812 00:09:23.677629 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.677687 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 00:09:23.678125 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.678016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49902)
0812 00:09:23.678070 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.678096 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 00:09:23.678212 332 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:42255: Network error: Client connection negotiation failed: client connection to 127.7.85.252:42255: connect: Connection refused (error 111)
I20260812 00:09:23.687512 487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:23.690550 487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:23.690747 487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:23.695350 487 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.252:42255
I20260812 00:09:23.695387 607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.252:42255 every 8 connection(s)
I20260812 00:09:23.695829 487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
I20260812 00:09:23.696838 608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:23.698879 608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.701421 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 487
I20260812 00:09:23.701555 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786493185917310-7511-0/minicluster-data/master-2/wal/instance
I20260812 00:09:23.701812 7511 external_mini_cluster.cc:1428] Setting key 724c00d23b39f339483adbde95579026
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:09:23.710266 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.699930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.7.85.254:41297)
0812 00:09:23.700001 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.700003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.700030 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:09:23.700619 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.700621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.700641 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.700750 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.701591 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.702299 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.703460 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.703692 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.703755 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.706891 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.706894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.706894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.706903 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.708551 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.709052 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.709053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.709053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.709191 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.709529 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.709531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.709531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.709704 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.709779 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.710061 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.710152 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 00:09:23.710335 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.699708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:41297 (local address 127.0.0.1:44588)
0812 00:09:23.699910 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.699955 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.700437 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.700883 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.700887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.701192 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.701417 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.701421 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.702399 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.702400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.703348 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.703350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.703793 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.703794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.703950 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.704699 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.704708 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.706780 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.708644 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.708654 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.708657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.708955 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.709314 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.709315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.709316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.709452 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.709778 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.709784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.709919 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.710011 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.710143 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":117,"threads_started":1}
I20260812 00:09:23.711246 608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.712782 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.702245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.7.85.252:42255)
0812 00:09:23.702464 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.702469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.702486 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:09:23.702506 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.702508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.702530 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.702617 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.703337 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.704287 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.705135 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.705711 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.705793 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.708692 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.708700 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.708700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.708708 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.710985 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.711855 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.711859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.711859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.711931 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.712172 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.712173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.712173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.712399 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.712553 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.712634 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.712697 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":112,"threads_started":1}
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:23.719774 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.711748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:38325 (local address 127.0.0.1:60514)
0812 00:09:23.711832 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.711843 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.711880 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.712081 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.712082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.712246 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.712295 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.712297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.713273 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.713274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.713954 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.713957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.714341 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.714343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.714404 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.714902 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.714907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.716214 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.718642 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.718643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.718644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.718841 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.719082 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.719085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.719085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.719144 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.719423 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.719424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.719476 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.719551 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.719661 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 00:09:23.719766 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.711863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.7.85.253:38325)
0812 00:09:23.711939 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.711941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.711948 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:09:23.711997 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.711998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.712010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.712070 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.712374 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.713146 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.714042 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.714260 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.714347 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.716312 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.716314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.716314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.716322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.718555 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.718932 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.718933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.718934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.719007 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.719218 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.719219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.719220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.719345 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.719420 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.719579 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.719661 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:09:23.720134 608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:09:23.723508 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.713926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44590 (local address 127.7.85.254:41297)
0812 00:09:23.713991 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.713994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.714003 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:09:23.714031 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.714032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.714044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.714108 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.714669 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.715358 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.717013 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.717376 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.717459 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.720380 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.720382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.720383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.720392 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.722074 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.722542 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.722544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.722544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.722605 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.722850 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.722851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.722852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.722982 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.723265 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.723308 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.723401 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:09:23.728570 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.720459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49930)
0812 00:09:23.720509 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.720521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.720561 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.720925 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.720926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.721049 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.721085 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.721086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.722167 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.722168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.722961 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.722964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.723470 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.723470 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.723514 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.723934 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.723940 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.725385 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.727257 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.727258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.727258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.727464 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.727754 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.727755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.727755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.727824 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.728154 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.728155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.728211 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.728301 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.728431 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 00:09:23.728742 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.720528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.7.85.252:42255)
0812 00:09:23.720588 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.720590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.720598 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:09:23.720620 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.720621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.720634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.720702 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.721230 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.722020 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.723063 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.723350 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.723423 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.725470 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.725472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.725472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.725480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.727141 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.727543 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.727544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.727545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.727625 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.727958 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.727959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.727959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.728077 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.728133 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.728337 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.728622 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":36}
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:09:23.730424 608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: Bootstrap starting.
I20260812 00:09:23.731666 608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:23.732046 608 log.cc:824] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:23.732923 608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: No bootstrap required, opened a new log
I20260812 00:09:23.733650 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.724319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.7.85.253:38325)
0812 00:09:23.724369 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.724371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.724474 (+ 103us) server_negotiation.cc:407] Connection header received
0812 00:09:23.724527 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.724528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.724539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.724615 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.725071 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.725806 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.726698 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.727010 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.727108 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.730256 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.730258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.730258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.730267 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.732340 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.732712 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.732714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.732714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.732774 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.733010 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.733011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.733011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.733140 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.733408 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.733448 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.733534 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 00:09:23.735112 608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.735275 608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:23.735277 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:09:23.735321 608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b5266df5ce480f96313f84f74be23f, State: Initialized, Role: FOLLOWER
I20260812 00:09:23.735453 608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.735782 615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.735850 608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:23.735909 615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:09:23.738557 651 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:09:23.738632 651 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:09:23.738788 487 master_runner.cc:186] Error getting master registration for 127.7.85.254:41297: OK, instance_id { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" instance_seqno: 1786493363327972 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:09:23.741086 487 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:09:23.748517 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.736964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.7.85.252:42255)
0812 00:09:23.737091 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.737093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.737551 (+ 458us) server_negotiation.cc:407] Connection header received
0812 00:09:23.737663 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.737664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.737676 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.737727 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.741628 (+ 3901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.742292 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.743367 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.743692 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.743734 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.745471 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.745473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.745474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.745482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.746925 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.747493 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.747495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.747495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.747560 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.747818 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.747819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.747819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.748006 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.748090 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.748315 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.748416 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 00:09:23.748593 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.737339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.7.85.253:38325)
0812 00:09:23.737380 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.737382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.737953 (+ 571us) server_negotiation.cc:407] Connection header received
0812 00:09:23.738710 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.738723 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.738736 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.738768 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.739408 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.740152 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.741585 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.741862 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.741919 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.743660 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.743662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.743662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.743671 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.745563 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.746250 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.746252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.746252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.746318 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.747176 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.747177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.747177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.747353 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.748092 (+ 739us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.748373 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.748484 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:09:23.749560 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.737193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.7.85.254:41297)
0812 00:09:23.737249 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.737251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.737386 (+ 135us) server_negotiation.cc:407] Connection header received
0812 00:09:23.737507 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.737508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.737522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.737557 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.739435 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.740227 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.741421 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.741699 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.741761 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.743692 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.743694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.743694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.743703 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.746409 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.747472 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.747474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.747475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.747538 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.747779 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.747780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.747780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.748000 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.748499 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.749404 (+ 905us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.749495 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 00:09:23.750324 476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } attempt: 3
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:09:23.758571 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.750707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.7.85.252:42255)
0812 00:09:23.750796 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.750798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.750805 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:09:23.750825 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.750825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.750836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.750887 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.751146 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.751915 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.753325 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.753589 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.753650 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.755871 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.755872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.755873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.755881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.757459 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.757859 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.757860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.757860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.757916 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.758115 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.758116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.758116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.758221 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.758272 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.758445 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.758505 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 00:09:23.758638 481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.750606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49948)
0812 00:09:23.750663 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.750676 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.750724 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.750904 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.750906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.751031 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.751071 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.751072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.752431 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.752432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.753245 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.753249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.753672 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.753673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.753743 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.754264 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.754269 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.755760 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.757538 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.757539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.757539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.757780 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.757984 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.757985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.757986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.758054 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.758282 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.758283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.758334 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.758410 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.758510 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:09:23.760115 476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b: Bootstrap starting.
I20260812 00:09:23.760792 476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:23.761181 476 log.cc:824] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:23.762101 476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b: No bootstrap required, opened a new log
I20260812 00:09:23.763655 476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.763811 476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:23.763868 476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 914084140d5b4fb69fcd293d141ff34b, State: Initialized, Role: FOLLOWER
I20260812 00:09:23.764038 476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.764356 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.764417 476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:23.764446 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:09:23.766753 671 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:09:23.766878 671 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:09:23.766871 352 master_runner.cc:186] Error getting master registration for 127.7.85.254:41297: OK, instance_id { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" instance_seqno: 1786493363327972 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:09:23.769405 352 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:09:23.777354 332 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } attempt: 3
Aug 12 00:09:23 dist-test-slave-z8x9 krb5kdc[32674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:09:23.786983 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.777705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:42255 (local address 127.0.0.1:49960)
0812 00:09:23.777760 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:23.777772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:23.777817 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:23.777984 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:23.777985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:23.778195 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:23.778262 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.778264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.779217 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.779218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.780036 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:23.780038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.780524 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.780525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:23.780587 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.781148 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:23.781154 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:23.783621 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:23.785750 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.785752 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.785752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.786008 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.786242 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:23.786243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:23.786243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:23.786318 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:23.786619 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:23.786620 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:23.786675 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 00:09:23.786750 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 00:09:23.786869 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 00:09:23.786983 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:23.777793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.7.85.252:42255)
0812 00:09:23.777842 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:23.777844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:23.777855 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:09:23.777884 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:23.777885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:23.777895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:23.777966 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:23.778349 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.779119 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.780129 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:23.780443 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:23.780515 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:23.783842 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:23.783844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:23.783844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:23.783853 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:23.785656 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.786091 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.786093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.786093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.786159 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:23.786395 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:23.786396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:23.786396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:23.786545 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:23.786615 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:23.786793 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 00:09:23.786880 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:09:23.789084 332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: Bootstrap starting.
I20260812 00:09:23.789969 332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:23.790277 332 log.cc:824] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:23.791141 332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: No bootstrap required, opened a new log
I20260812 00:09:23.793445 332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.793603 332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:23.793637 332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f76e8bd69ee4cd18b5fd434396a86ad, State: Initialized, Role: FOLLOWER
I20260812 00:09:23.793794 332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.794153 332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: configured and running, proceeding with master startup.
W20260812 00:09:23.796056 695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:09:23.796182 695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:09:23.794096 680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.796327 680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:09:23.798514 32699 master_runner.cc:263] No leader master found from master 3f76e8bd69ee4cd18b5fd434396a86ad
I20260812 00:09:23.798565 32699 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 00:09:23.825670 32730 tablet.cc:2369] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:09:23.828694 656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:23.828997 656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.829309 656 leader_election.cc:288] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f76e8bd69ee4cd18b5fd434396a86ad (127.7.85.254:41297), 09b5266df5ce480f96313f84f74be23f (127.7.85.252:42255)
I20260812 00:09:23.829854 32761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "914084140d5b4fb69fcd293d141ff34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" is_pre_election: true
I20260812 00:09:23.830022 32761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 914084140d5b4fb69fcd293d141ff34b in term 0.
I20260812 00:09:23.830474 560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "914084140d5b4fb69fcd293d141ff34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b5266df5ce480f96313f84f74be23f" is_pre_election: true
I20260812 00:09:23.830629 560 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 914084140d5b4fb69fcd293d141ff34b in term 0.
I20260812 00:09:23.830848 377 leader_election.cc:302] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f76e8bd69ee4cd18b5fd434396a86ad, 914084140d5b4fb69fcd293d141ff34b; no voters:
I20260812 00:09:23.831905 656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:09:23.831982 656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:09:23.832007 656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:23.833926 656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.834095 656 leader_election.cc:288] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [CANDIDATE]: Term 1 election: Requested vote from peers 3f76e8bd69ee4cd18b5fd434396a86ad (127.7.85.254:41297), 09b5266df5ce480f96313f84f74be23f (127.7.85.252:42255)
I20260812 00:09:23.834570 32761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "914084140d5b4fb69fcd293d141ff34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad"
I20260812 00:09:23.834676 32761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:23.835676 32761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 914084140d5b4fb69fcd293d141ff34b in term 1.
I20260812 00:09:23.835963 377 leader_election.cc:302] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f76e8bd69ee4cd18b5fd434396a86ad, 914084140d5b4fb69fcd293d141ff34b; no voters:
I20260812 00:09:23.836093 560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "914084140d5b4fb69fcd293d141ff34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b5266df5ce480f96313f84f74be23f"
I20260812 00:09:23.836198 560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:23.837412 560 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 914084140d5b4fb69fcd293d141ff34b in term 1.
I20260812 00:09:23.837685 656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:23.837870 656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [term 1 LEADER]: Becoming Leader. State: Replica: 914084140d5b4fb69fcd293d141ff34b, State: Running, Role: LEADER
I20260812 00:09:23.837970 656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } }
I20260812 00:09:23.838369 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 914084140d5b4fb69fcd293d141ff34b. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.838435 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:23.838765 703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:23.838989 703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:23.841665 560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [term 1 FOLLOWER]: Refusing update from remote peer 914084140d5b4fb69fcd293d141ff34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:23.841964 32761 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [term 1 FOLLOWER]: Refusing update from remote peer 914084140d5b4fb69fcd293d141ff34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:23.842425 656 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:23.843803 615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 914084140d5b4fb69fcd293d141ff34b. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.843918 615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:09:23.844198 656 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:23.845589 680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 914084140d5b4fb69fcd293d141ff34b. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.845708 680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:09:23.847579 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.847673 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:23.849308 680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.849408 680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:09:23.849529 705 mvcc.cc:205] Tried to move back new op lower bound from 7317476818290622464 to 7317476818280767488. Current Snapshot: MvccSnapshot[applied={T|T < 7317476818290622464}]
I20260812 00:09:23.850100 717 mvcc.cc:205] Tried to move back new op lower bound from 7317476818290622464 to 7317476818280767488. Current Snapshot: MvccSnapshot[applied={T|T < 7317476818290622464}]
I20260812 00:09:23.850595 656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.850684 656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:23.851142 615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "914084140d5b4fb69fcd293d141ff34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f76e8bd69ee4cd18b5fd434396a86ad" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41297 } } peers { permanent_uuid: "914084140d5b4fb69fcd293d141ff34b" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 38325 } } peers { permanent_uuid: "09b5266df5ce480f96313f84f74be23f" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 42255 } } }
I20260812 00:09:23.851249 615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:09:23.851961 710 mvcc.cc:205] Tried to move back new op lower bound from 7317476818290622464 to 7317476818280767488. Current Snapshot: MvccSnapshot[applied={T|T < 7317476818290622464}]
I20260812 00:09:23.852916 703 catalog_manager.cc:1347] Generated new cluster ID: 7f2b771430684584bbbad3618e843eb3
I20260812 00:09:23.852965 703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:09:23.886338 703 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:09:23.887212 703 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:23.911563 703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 914084140d5b4fb69fcd293d141ff34b: Generated new TSK 0
I20260812 00:09:23.911933 703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:09:24.739293 651 catalog_manager.cc:1259] Loaded cluster ID: 7f2b771430684584bbbad3618e843eb3
I20260812 00:09:24.739391 651 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: loading cluster ID for follower catalog manager: success
I20260812 00:09:24.740329 651 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: acquiring CA information for follower catalog manager: success
I20260812 00:09:24.740701 651 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09b5266df5ce480f96313f84f74be23f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:09:24.797034 695 catalog_manager.cc:1259] Loaded cluster ID: 7f2b771430684584bbbad3618e843eb3
I20260812 00:09:24.797144 695 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: loading cluster ID for follower catalog manager: success
I20260812 00:09:24.798110 695 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: acquiring CA information for follower catalog manager: success
I20260812 00:09:24.798445 695 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f76e8bd69ee4cd18b5fd434396a86ad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:09:24.971199 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 32699
I20260812 00:09:24.987752 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 352
I20260812 00:09:25.009675 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 487
2026-08-12T00:09:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3409 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:09:25 dist-test-slave-z8x9 krb5kdc[736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:09:25 dist-test-slave-z8x9 krb5kdc[736](info): set up 2 sockets
Aug 12 00:09:25 dist-test-slave-z8x9 krb5kdc[736](info): commencing operation
krb5kdc: starting...
W20260812 00:09:26.376416 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:09:26 dist-test-slave-z8x9 krb5kdc[736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:09:26.526667 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:45219
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:45219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:26.615511 754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:26.615799 754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:26.615842 754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:26.617028 754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:26.617102 754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:26.617127 754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:26.617218 754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:26.617250 754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:26.618880 754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:45219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:45219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:26.619184 754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:26.619434 754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:26.624950 766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:26.625104 754 server_base.cc:1034] running on GCE node
W20260812 00:09:26.624972 762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:26.624959 763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:26.625550 754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:26.625842 754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:26.627018 754 hybrid_clock.cc:648] HybridClock initialized: now 1786493366626990 us; error 45 us; skew 500 ppm
Aug 12 00:09:26 dist-test-slave-z8x9 krb5kdc[736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493366, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:26.629813 754 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:26.630292 754 webserver.cc:460] Webserver started at http://127.7.85.254:45233/ using document root <none> and password file <none>
I20260812 00:09:26.630561 754 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:26.630611 754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:26.630755 754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:26.631896 754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "1b419ab713254a2e83fd164b9a35e6e8"
format_stamp: "Formatted at 2026-08-12 00:09:26 on dist-test-slave-z8x9"
server_key: "0cdc353665b08c96b1a864b4db053811"
server_key_iv: "38fdb479f4f2a37916910fb2cf923c7b"
server_key_version: "encryptionkey@0"
I20260812 00:09:26.632266 754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "1b419ab713254a2e83fd164b9a35e6e8"
format_stamp: "Formatted at 2026-08-12 00:09:26 on dist-test-slave-z8x9"
server_key: "0cdc353665b08c96b1a864b4db053811"
server_key_iv: "38fdb479f4f2a37916910fb2cf923c7b"
server_key_version: "encryptionkey@0"
I20260812 00:09:26.633880 754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:09:26.634783 776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:26.635046 754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:26.635155 754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "1b419ab713254a2e83fd164b9a35e6e8"
format_stamp: "Formatted at 2026-08-12 00:09:26 on dist-test-slave-z8x9"
server_key: "0cdc353665b08c96b1a864b4db053811"
server_key_iv: "38fdb479f4f2a37916910fb2cf923c7b"
server_key_version: "encryptionkey@0"
I20260812 00:09:26.635349 754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:26.650588 754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:26.653419 754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:26.653568 754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:26.658166 754 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:45219
I20260812 00:09:26.658207 866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:45219 every 8 connection(s)
I20260812 00:09:26.658597 754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:26.659590 867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:26.662070 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8: Bootstrap starting.
I20260812 00:09:26.663067 867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:26.663446 867 log.cc:824] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:26.663666 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 754
I20260812 00:09:26.663782 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:09:26.663990 7511 external_mini_cluster.cc:1428] Setting key 26f61f1c4f9aa6bc9b824e9ef12f123b
I20260812 00:09:26.665144 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8: No bootstrap required, opened a new log
I20260812 00:09:26.668241 867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b419ab713254a2e83fd164b9a35e6e8" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45219 } }
I20260812 00:09:26.668437 867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:26.668493 867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b419ab713254a2e83fd164b9a35e6e8, State: Initialized, Role: FOLLOWER
I20260812 00:09:26.668646 867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b419ab713254a2e83fd164b9a35e6e8" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45219 } }
I20260812 00:09:26.668742 867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:26.668787 867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:26.668843 867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:26.669684 867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b419ab713254a2e83fd164b9a35e6e8" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45219 } }
I20260812 00:09:26.669817 867 leader_election.cc:302] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b419ab713254a2e83fd164b9a35e6e8; no voters:
I20260812 00:09:26.670035 867 leader_election.cc:288] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:09:26.670091 872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:26.670281 872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [term 1 LEADER]: Becoming Leader. State: Replica: 1b419ab713254a2e83fd164b9a35e6e8, State: Running, Role: LEADER
I20260812 00:09:26.670367 867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:26.670411 872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b419ab713254a2e83fd164b9a35e6e8" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45219 } }
I20260812 00:09:26.670836 872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b419ab713254a2e83fd164b9a35e6e8. Latest consensus state: current_term: 1 leader_uuid: "1b419ab713254a2e83fd164b9a35e6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b419ab713254a2e83fd164b9a35e6e8" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45219 } } }
I20260812 00:09:26.670841 876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b419ab713254a2e83fd164b9a35e6e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b419ab713254a2e83fd164b9a35e6e8" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45219 } } }
I20260812 00:09:26.670948 872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:26.671020 876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:26.671233 883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:26.671787 883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:26.673710 883 catalog_manager.cc:1347] Generated new cluster ID: 1a11d2b1569d4d908c444ccfe6ba78a4
I20260812 00:09:26.673763 883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:09:26 dist-test-slave-z8x9 krb5kdc[736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:26.678794 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:26.664646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.7.85.254:45219)
0812 00:09:26.665400 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:26.665410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:26.665447 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:09:26.666918 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:26.666927 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:26.666984 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:26.667115 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:26.668049 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:26.669501 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:26.670717 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:26.671403 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:26.671530 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:26.674816 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:26.674821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:26.674822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:26.674837 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:26.676884 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:26.677431 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:26.677435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:26.677435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:26.677538 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:26.677783 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:26.677784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:26.677785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:26.678063 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:26.678264 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:26.678396 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 00:09:26.678559 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":100,"threads_started":1}
I20260812 00:09:26.679852 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:09:26.686100 883 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:09:26.686823 883 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:26.691924 883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b419ab713254a2e83fd164b9a35e6e8: Generated new TSK 0
I20260812 00:09:26.692158 883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:09:26.735044 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 754
2026-08-12T00:09:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1733 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:09:26 dist-test-slave-z8x9 krb5kdc[908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:09:26 dist-test-slave-z8x9 krb5kdc[908](info): set up 2 sockets
Aug 12 00:09:26 dist-test-slave-z8x9 krb5kdc[908](info): commencing operation
krb5kdc: starting...
W20260812 00:09:28.092317 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:09:28.239048 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33245
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:28.331403 931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:28.331648 931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:28.331686 931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:28.332873 931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:28.332916 931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:28.332931 931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:28.332944 931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:28.332957 931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:28.334723 931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:28.334971 931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:28.335162 931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:28.340613 939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.340611 952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.340890 942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:28.340950 931 server_base.cc:1034] running on GCE node
I20260812 00:09:28.341329 931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:28.341660 931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:28.342979 931 hybrid_clock.cc:648] HybridClock initialized: now 1786493368342969 us; error 67 us; skew 500 ppm
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:28.352450 931 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:28.353336 931 webserver.cc:460] Webserver started at http://127.7.85.254:34953/ using document root <none> and password file <none>
I20260812 00:09:28.353574 931 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:28.353627 931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:28.353729 931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:28.354868 931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "cb7707ba261c4082940d9ab37a866108"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "085d9da81389e631944d963d2220a190"
server_key_iv: "4df886f9ba670b18fb5c71f666909c91"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.355212 931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "cb7707ba261c4082940d9ab37a866108"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "085d9da81389e631944d963d2220a190"
server_key_iv: "4df886f9ba670b18fb5c71f666909c91"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.356933 931 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:09:28.358139 959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.358449 931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:28.358558 931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "cb7707ba261c4082940d9ab37a866108"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "085d9da81389e631944d963d2220a190"
server_key_iv: "4df886f9ba670b18fb5c71f666909c91"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.358630 931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:28.375741 931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:28.379403 931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:28.379563 931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:28.383951 931 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33245
I20260812 00:09:28.384040 1041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33245 every 8 connection(s)
I20260812 00:09:28.384411 931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:28.385471 1042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:28.386691 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 931
I20260812 00:09:28.386793 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:09:28.387140 7511 external_mini_cluster.cc:1428] Setting key 2277b78239a3cc1bbe67bc17080a8bba
I20260812 00:09:28.388283 1042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap starting.
I20260812 00:09:28.388947 1042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:28.389319 1042 log.cc:824] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:28.390234 1042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: No bootstrap required, opened a new log
I20260812 00:09:28.392167 1042 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:28.392347 1042 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:28.392397 1042 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb7707ba261c4082940d9ab37a866108, State: Initialized, Role: FOLLOWER
I20260812 00:09:28.392586 1042 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:28.392709 1042 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:28.392759 1042 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:28.392815 1042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:28.393532 1042 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:28.393677 1042 leader_election.cc:302] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb7707ba261c4082940d9ab37a866108; no voters:
I20260812 00:09:28.393920 1042 leader_election.cc:288] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:09:28.393945 1050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:28.394071 1050 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 LEADER]: Becoming Leader. State: Replica: cb7707ba261c4082940d9ab37a866108, State: Running, Role: LEADER
I20260812 00:09:28.394232 1042 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:28.394234 1050 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:28.394804 1052 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb7707ba261c4082940d9ab37a866108. Latest consensus state: current_term: 1 leader_uuid: "cb7707ba261c4082940d9ab37a866108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } } }
I20260812 00:09:28.394987 1052 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:28.394811 1051 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb7707ba261c4082940d9ab37a866108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } } }
I20260812 00:09:28.395530 1051 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: This master's current role is: LEADER
W20260812 00:09:28.396600 1071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:09:28.396737 1071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:09:28.397053 1061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:28.397539 1061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:28.399703 1061 catalog_manager.cc:1347] Generated new cluster ID: 12e95ae5f512491a8f8a226900523bf7
I20260812 00:09:28.399782 1061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:09:28.401692 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.387791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.7.85.254:33245)
0812 00:09:28.387997 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:28.388006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:28.388032 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:09:28.388526 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:28.388533 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:28.388594 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:28.388748 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:28.391023 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.392039 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.393857 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.394506 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.394633 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:28.398023 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:28.398028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:28.398028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:28.398041 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:28.400046 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:28.400450 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:28.400453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:28.400453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:28.400545 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:28.400771 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:28.400772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:28.400772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:28.401023 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:28.401253 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:28.401350 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 00:09:28.401502 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260812 00:09:28.423086 1061 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:09:28.424111 1061 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:28.426251 1061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Generated new TSK 0
I20260812 00:09:28.426502 1061 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:09:28.470537 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33245
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:28.573266 1079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:28.573511 1079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:28.573540 1079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:28.574764 1079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:28.574807 1079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:28.574834 1079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:09:28.576732 1079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:28.577090 1079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:28.577409 1079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:28.583508 1087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.583508 1094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.583527 1088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:28.583998 1079 server_base.cc:1034] running on GCE node
I20260812 00:09:28.584273 1079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:28.584519 1079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:28.585721 1079 hybrid_clock.cc:648] HybridClock initialized: now 1786493368585690 us; error 40 us; skew 500 ppm
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:28.588747 1079 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:09:28.589344 1079 webserver.cc:460] Webserver started at http://127.7.85.193:43765/ using document root <none> and password file <none>
I20260812 00:09:28.589612 1079 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:28.589687 1079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:28.589854 1079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:28.591009 1079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "de9e8fbebb704a8293f078551f000192"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "59521535a7c6b46c68305d9c21ebd702"
server_key_iv: "c2481898e32b34ba8cac1deca6378642"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.591408 1079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "de9e8fbebb704a8293f078551f000192"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "59521535a7c6b46c68305d9c21ebd702"
server_key_iv: "c2481898e32b34ba8cac1deca6378642"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.593415 1079 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:09:28.595070 1103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.595393 1079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:09:28.595600 1079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "de9e8fbebb704a8293f078551f000192"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "59521535a7c6b46c68305d9c21ebd702"
server_key_iv: "c2481898e32b34ba8cac1deca6378642"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.595714 1079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:28.614022 1079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:28.616787 1079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:28.616969 1079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:28.617304 1079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:28.617662 1079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:28.617722 1079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.617761 1079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:28.617789 1079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.624238 1079 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:41713
I20260812 00:09:28.624289 1300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:41713 every 8 connection(s)
I20260812 00:09:28.624663 1079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:09:28.628894 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 1079
I20260812 00:09:28.629042 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:09:28.629345 7511 external_mini_cluster.cc:1428] Setting key 73783f1f8dec9e46421a77b60bc1fd28
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:28.636029 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.625598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:50411 (local address 127.7.85.254:33245)
0812 00:09:28.625662 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:28.625665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:28.625842 (+ 177us) server_negotiation.cc:407] Connection header received
0812 00:09:28.626341 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:28.626342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:28.626352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:28.626421 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:28.627442 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.628003 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.629738 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.629969 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.630043 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:28.632895 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:28.632897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:28.632897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:28.632913 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:28.634401 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:28.634893 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:28.634894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:28.634895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:28.634989 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:28.635279 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:28.635280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:28.635280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:28.635415 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:28.635492 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:28.635816 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 00:09:28.635925 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:09:28.636282 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.625541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:50411)
0812 00:09:28.625709 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:28.625754 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:28.626258 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:28.626476 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:28.626480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:28.627086 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:28.627309 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:28.627314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.628224 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.628227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:28.629619 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:28.629626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.630069 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.630070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:28.630318 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:28.631001 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:28.631010 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:28.632778 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:28.634502 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:28.634513 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:28.634516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:28.634794 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:28.635063 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:28.635063 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:28.635064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:28.635201 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:28.635492 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:28.635502 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:28.635673 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 00:09:28.635793 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 00:09:28.635990 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260812 00:09:28.637321 1303 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33245
I20260812 00:09:28.637563 1303 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:28.637883 1303 heartbeater.cc:511] Master 127.7.85.254:33245 requested a full tablet report, sending...
I20260812 00:09:28.638819 986 ts_manager.cc:194] Registered new tserver with Master: de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713)
I20260812 00:09:28.639842 986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:50411
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:09:28.661299 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33245
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:28.753944 1309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:28.754186 1309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:28.754215 1309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:28.755441 1309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:28.755493 1309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:28.755522 1309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:09:28.757079 1309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:28.757387 1309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:28.757655 1309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:28.763250 1316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.763222 1319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.763226 1315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:28.763330 1309 server_base.cc:1034] running on GCE node
I20260812 00:09:28.763794 1309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:28.764117 1309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:28.765295 1309 hybrid_clock.cc:648] HybridClock initialized: now 1786493368765258 us; error 56 us; skew 500 ppm
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:28.768076 1309 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:09:28.768509 1309 webserver.cc:460] Webserver started at http://127.7.85.194:39803/ using document root <none> and password file <none>
I20260812 00:09:28.768766 1309 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:28.768817 1309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:28.768950 1309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:28.770112 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "8e66ebc7ed4e4285821aeab116a13368"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "09938fac72808034e8c81a01243d682b"
server_key_iv: "7492aa16e713e5f083b12abd22cf6d45"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.770493 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "8e66ebc7ed4e4285821aeab116a13368"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "09938fac72808034e8c81a01243d682b"
server_key_iv: "7492aa16e713e5f083b12abd22cf6d45"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.772361 1309 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:09:28.773352 1329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.773646 1309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:28.773739 1309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "8e66ebc7ed4e4285821aeab116a13368"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "09938fac72808034e8c81a01243d682b"
server_key_iv: "7492aa16e713e5f083b12abd22cf6d45"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.773841 1309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:28.801748 1309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:28.804138 1309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:28.804314 1309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:28.804597 1309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:28.804943 1309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:28.804980 1309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.805003 1309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:28.805016 1309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.811283 1309 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35467
I20260812 00:09:28.811332 1519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35467 every 8 connection(s)
I20260812 00:09:28.811866 1309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:28.820246 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 1309
I20260812 00:09:28.820488 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:09:28.820724 7511 external_mini_cluster.cc:1428] Setting key 23b9a58658aaaa1ec2e2302b0e174201
I20260812 00:09:28.822971 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.812817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:35819 (local address 127.7.85.254:33245)
0812 00:09:28.812949 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:28.812952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:28.813236 (+ 284us) server_negotiation.cc:407] Connection header received
0812 00:09:28.813824 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:28.813825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:28.813839 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:28.813895 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:28.814931 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.815455 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.816499 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.816700 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.816780 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:28.819310 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:28.819312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:28.819312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:28.819322 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:28.821102 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:28.821798 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:28.821800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:28.821800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:28.821874 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:28.822208 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:28.822209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:28.822209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:28.822338 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:28.822404 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:28.822810 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 00:09:28.822897 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260812 00:09:28.823278 1529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.812730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:35819)
0812 00:09:28.812986 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:28.813098 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:28.813703 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:28.813967 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:28.813974 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:28.814610 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:28.814825 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:28.814830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.815569 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.815571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:28.816400 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:28.816402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.816796 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.816797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:28.816933 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:28.817588 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:28.817597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:28.819179 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:28.821240 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:28.821252 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:28.821256 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:28.821689 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:28.821955 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:28.821956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:28.821957 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:28.822125 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:28.822416 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:28.822424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:28.822630 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 00:09:28.822758 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 00:09:28.822961 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260812 00:09:28.823871 1526 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33245
I20260812 00:09:28.823980 1526 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:28.824211 1526 heartbeater.cc:511] Master 127.7.85.254:33245 requested a full tablet report, sending...
I20260812 00:09:28.824647 986 ts_manager.cc:194] Registered new tserver with Master: 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
I20260812 00:09:28.825062 986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:35819
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:09:28.857743 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33245
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:28.948098 1534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:28.948508 1534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:28.948578 1534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:28.949996 1534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:28.950049 1534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:28.950079 1534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:09:28.951608 1534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:28.951853 1534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:28.952054 1534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:28.957813 1545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.957803 1544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:28.957803 1547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:28.958082 1534 server_base.cc:1034] running on GCE node
I20260812 00:09:28.958302 1534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:28.958590 1534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:28.959803 1534 hybrid_clock.cc:648] HybridClock initialized: now 1786493368959782 us; error 31 us; skew 500 ppm
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:28.962942 1534 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:09:28.963469 1534 webserver.cc:460] Webserver started at http://127.7.85.195:41607/ using document root <none> and password file <none>
I20260812 00:09:28.963701 1534 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:28.963840 1534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:28.963997 1534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:28.965088 1534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "8d3a413bfa7f49ebaf90aff344964e83"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "bc3c0e130a4c2e091787ed2c21e46f07"
server_key_iv: "392bf141e97c742d3162b85007e466cd"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.965502 1534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "8d3a413bfa7f49ebaf90aff344964e83"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "bc3c0e130a4c2e091787ed2c21e46f07"
server_key_iv: "392bf141e97c742d3162b85007e466cd"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.966902 1534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:28.967842 1566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.968050 1534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:28.968145 1534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "8d3a413bfa7f49ebaf90aff344964e83"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "bc3c0e130a4c2e091787ed2c21e46f07"
server_key_iv: "392bf141e97c742d3162b85007e466cd"
server_key_version: "encryptionkey@0"
I20260812 00:09:28.968211 1534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:28.981819 1534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:28.984318 1534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:28.984449 1534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:28.984720 1534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:28.985076 1534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:28.985114 1534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.985138 1534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:28.985152 1534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:28.991677 1534 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34141
I20260812 00:09:28.991753 1774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34141 every 8 connection(s)
I20260812 00:09:28.992143 1534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:09:28.994871 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 1534
I20260812 00:09:28.994966 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:09:28.995173 7511 external_mini_cluster.cc:1428] Setting key 96162439206604233dadc7060bce452d
Aug 12 00:09:28 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:29.005707 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.993132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:50121 (local address 127.7.85.254:33245)
0812 00:09:28.993212 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:28.993215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:28.993549 (+ 334us) server_negotiation.cc:407] Connection header received
0812 00:09:28.994113 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:28.994114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:28.994126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:28.994200 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:28.996074 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.996915 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.998371 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.998610 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.998728 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.002147 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:29.002151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:29.002151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:29.002169 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:29.003988 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.004516 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.004517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.004518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.004583 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.004877 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.004878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.004879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.005019 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:29.005113 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.005504 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.005602 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:09:29.005959 1777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:28.993102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:50121)
0812 00:09:28.993408 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:28.993455 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:28.993989 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:28.994486 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:28.994495 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:28.995543 (+ 1048us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:28.995948 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:28.995965 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.997075 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.997077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:28.998238 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:28.998243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:28.998763 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:28.998765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:28.998989 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.000004 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:29.000018 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:29.001995 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:29.004125 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.004137 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.004140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.004417 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.004679 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.004680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.004680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.004798 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.005114 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:29.005124 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:29.005354 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.005469 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.005678 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":183,"threads_started":1}
I20260812 00:09:29.006856 1775 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33245
I20260812 00:09:29.007004 1775 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:29.007344 1775 heartbeater.cc:511] Master 127.7.85.254:33245 requested a full tablet report, sending...
I20260812 00:09:29.008388 986 ts_manager.cc:194] Registered new tserver with Master: 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141)
I20260812 00:09:29.008831 986 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:50121
I20260812 00:09:29.019462 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:09:29.029081 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.020984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42730 (local address 127.7.85.254:33245)
0812 00:09:29.021079 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.021081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.021243 (+ 162us) server_negotiation.cc:407] Connection header received
0812 00:09:29.021279 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.021280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.021292 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.021348 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:29.021973 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.022576 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.023386 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.023747 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.023830 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.025416 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:29.025417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:29.025418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:29.025426 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:29.027417 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.028044 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.028045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.028046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.028130 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.028376 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.028377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.028377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.028554 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:29.028650 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.028920 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.029007 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:09:29.031579 986 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:09:29.033469 986 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:09:29.043021 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.038136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35467 (local address 127.0.0.1:36514)
0812 00:09:29.038318 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.038333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.038418 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.038855 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.038856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.038859 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.038912 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.038914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.040399 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.040401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.041511 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.041514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.042716 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.042717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.042794 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.042798 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.042863 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.042908 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":33,"thread_start_us":101,"threads_started":1}
I20260812 00:09:29.043042 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.038368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.7.85.194:35467)
0812 00:09:29.038652 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.038656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.038668 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:09:29.038715 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.038717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.038734 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.038992 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.039117 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.040240 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.041625 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.042491 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.042718 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.042805 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.042893 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.042946 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":142,"threads_started":1}
I20260812 00:09:29.043406 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.037931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.7.85.193:41713)
0812 00:09:29.038165 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.038176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.038362 (+ 186us) server_negotiation.cc:407] Connection header received
0812 00:09:29.038507 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.038510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.038528 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.038591 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.038984 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.039749 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.041158 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.042231 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.042416 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.042518 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.042967 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.043028 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
I20260812 00:09:29.042900 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.037777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:41713 (local address 127.0.0.1:60612)
0812 00:09:29.038272 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.038287 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.038395 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.038722 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.038724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.038727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.038866 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.038870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.039858 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.039859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.041055 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.041058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.042447 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.042449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.042564 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.042581 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.042697 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.042770 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":162,"threads_started":1}
I20260812 00:09:29.043563 1800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.037704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34141 (local address 127.0.0.1:59044)
0812 00:09:29.037944 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.037992 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.038086 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.038713 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.038719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.038722 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.038864 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.038868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.040417 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.040418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.041830 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.041835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.043235 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.043237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.043347 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.043352 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.043404 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.043482 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260812 00:09:29.043879 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.037886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.7.85.195:34141)
0812 00:09:29.038377 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.038385 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.038399 (+ 14us) server_negotiation.cc:407] Connection header received
0812 00:09:29.038463 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.038466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.038487 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.038586 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.038962 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.040241 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.041973 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.043052 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.043134 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.043196 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.043743 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.043798 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":126,"threads_started":1}
I20260812 00:09:29.044034 1408 tablet_service.cc:1467] Processing CreateTablet for tablet bcf429a3e861461bba4fac835f305f24 (DEFAULT_TABLE table=test-table [id=91a37d93021e466fa0a666b6087fa124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:29.044459 1408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf429a3e861461bba4fac835f305f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:29.044575 1644 tablet_service.cc:1467] Processing CreateTablet for tablet bcf429a3e861461bba4fac835f305f24 (DEFAULT_TABLE table=test-table [id=91a37d93021e466fa0a666b6087fa124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:29.044872 1644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf429a3e861461bba4fac835f305f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:29.047257 1196 tablet_service.cc:1467] Processing CreateTablet for tablet bcf429a3e861461bba4fac835f305f24 (DEFAULT_TABLE table=test-table [id=91a37d93021e466fa0a666b6087fa124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:29.047628 1196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcf429a3e861461bba4fac835f305f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:29.048688 1815 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Bootstrap starting.
I20260812 00:09:29.049299 1812 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Bootstrap starting.
I20260812 00:09:29.049477 1815 tablet_bootstrap.cc:654] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:29.049947 1812 tablet_bootstrap.cc:654] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:29.050139 1815 log.cc:824] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.050585 1812 log.cc:824] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.051086 1815 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: No bootstrap required, opened a new log
I20260812 00:09:29.051254 1815 ts_tablet_manager.cc:1397] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:09:29.051414 1820 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Bootstrap starting.
I20260812 00:09:29.051487 1812 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: No bootstrap required, opened a new log
I20260812 00:09:29.051549 1812 ts_tablet_manager.cc:1397] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:09:29.052105 1820 tablet_bootstrap.cc:654] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:29.052706 1820 log.cc:824] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.053915 1820 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: No bootstrap required, opened a new log
I20260812 00:09:29.053802 1812 raft_consensus.cc:348] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.054003 1820 ts_tablet_manager.cc:1397] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:09:29.053987 1812 raft_consensus.cc:374] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:29.054055 1812 raft_consensus.cc:729] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d3a413bfa7f49ebaf90aff344964e83, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.053941 1815 raft_consensus.cc:348] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.054220 1812 consensus_queue.cc:260] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.054250 1815 raft_consensus.cc:374] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:29.054318 1815 raft_consensus.cc:729] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e66ebc7ed4e4285821aeab116a13368, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.054529 1812 ts_tablet_manager.cc:1428] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:09:29.054680 1775 heartbeater.cc:503] Master 127.7.85.254:33245 was elected leader, sending a full tablet report...
I20260812 00:09:29.054636 1815 consensus_queue.cc:260] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.055073 1526 heartbeater.cc:503] Master 127.7.85.254:33245 was elected leader, sending a full tablet report...
I20260812 00:09:29.055727 1815 ts_tablet_manager.cc:1428] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 00:09:29.056432 1820 raft_consensus.cc:348] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.056608 1820 raft_consensus.cc:374] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:29.056658 1820 raft_consensus.cc:729] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9e8fbebb704a8293f078551f000192, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.056784 1820 consensus_queue.cc:260] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.057096 1820 ts_tablet_manager.cc:1428] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:09:29.057114 1303 heartbeater.cc:503] Master 127.7.85.254:33245 was elected leader, sending a full tablet report...
W20260812 00:09:29.062589 1527 tablet.cc:2369] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:09:29.072597 1826 raft_consensus.cc:482] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:29.072769 1826 raft_consensus.cc:504] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.073240 1826 leader_election.cc:288] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713), 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141)
I20260812 00:09:29.076807 1529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.073445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:41713 (local address 127.7.85.194:33995)
0812 00:09:29.073537 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.073575 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.073629 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.073913 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.073915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.073917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.074016 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.074018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.074890 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.074891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.075892 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.075895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.076504 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.076505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.076589 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.076593 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.076656 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.076727 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 00:09:29.076834 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.073609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:33995 (local address 127.7.85.193:41713)
0812 00:09:29.073708 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.073711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.073720 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:09:29.073753 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.073774 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.073790 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.073853 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.074159 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.074801 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.075972 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.076428 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.076485 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.076493 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.076695 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.076751 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:09:29.077018 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.073730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:50059 (local address 127.7.85.195:34141)
0812 00:09:29.073819 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.073822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.073955 (+ 133us) server_negotiation.cc:407] Connection header received
0812 00:09:29.074000 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.074001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.074014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.074081 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.074387 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.074889 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.076156 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.076614 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.076691 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.076699 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.076888 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.076933 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:09:29.077303 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.073607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34141 (local address 127.7.85.194:50059)
0812 00:09:29.073860 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.073871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.073949 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.074166 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.074168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.074172 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.074281 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.074283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.074994 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.074995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.076051 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.076055 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.076728 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.076729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.076804 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.076808 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.077161 (+ 353us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.077228 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":170,"threads_started":1}
I20260812 00:09:29.077499 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf429a3e861461bba4fac835f305f24" candidate_uuid: "8e66ebc7ed4e4285821aeab116a13368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9e8fbebb704a8293f078551f000192" is_pre_election: true
I20260812 00:09:29.077702 1229 raft_consensus.cc:2454] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e66ebc7ed4e4285821aeab116a13368 in term 0.
I20260812 00:09:29.077690 1682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf429a3e861461bba4fac835f305f24" candidate_uuid: "8e66ebc7ed4e4285821aeab116a13368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d3a413bfa7f49ebaf90aff344964e83" is_pre_election: true
I20260812 00:09:29.077862 1682 raft_consensus.cc:2454] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e66ebc7ed4e4285821aeab116a13368 in term 0.
I20260812 00:09:29.077936 1333 leader_election.cc:302] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e66ebc7ed4e4285821aeab116a13368, de9e8fbebb704a8293f078551f000192; no voters:
I20260812 00:09:29.078081 1826 raft_consensus.cc:2781] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:09:29.078135 1826 raft_consensus.cc:482] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:09:29.078150 1826 raft_consensus.cc:3037] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:29.078938 1826 raft_consensus.cc:504] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.079094 1826 leader_election.cc:288] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [CANDIDATE]: Term 1 election: Requested vote from peers de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713), 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141)
I20260812 00:09:29.079280 1682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf429a3e861461bba4fac835f305f24" candidate_uuid: "8e66ebc7ed4e4285821aeab116a13368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d3a413bfa7f49ebaf90aff344964e83"
I20260812 00:09:29.079305 1229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcf429a3e861461bba4fac835f305f24" candidate_uuid: "8e66ebc7ed4e4285821aeab116a13368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9e8fbebb704a8293f078551f000192"
I20260812 00:09:29.079371 1229 raft_consensus.cc:3037] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:29.079371 1682 raft_consensus.cc:3037] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:29.080283 1682 raft_consensus.cc:2454] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e66ebc7ed4e4285821aeab116a13368 in term 1.
I20260812 00:09:29.080283 1229 raft_consensus.cc:2454] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e66ebc7ed4e4285821aeab116a13368 in term 1.
I20260812 00:09:29.080461 1335 leader_election.cc:302] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d3a413bfa7f49ebaf90aff344964e83, 8e66ebc7ed4e4285821aeab116a13368; no voters:
I20260812 00:09:29.080572 1826 raft_consensus.cc:2781] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:29.080739 1826 raft_consensus.cc:686] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 1 LEADER]: Becoming Leader. State: Replica: 8e66ebc7ed4e4285821aeab116a13368, State: Running, Role: LEADER
I20260812 00:09:29.080813 1826 consensus_queue.cc:237] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.081676 985 catalog_manager.cc:5495] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194). New cstate: current_term: 1 leader_uuid: "8e66ebc7ed4e4285821aeab116a13368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } health_report { overall_health: HEALTHY } } }
I20260812 00:09:29.089645 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.087178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.7.85.194:35467)
0812 00:09:29.087227 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.087229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.087240 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:09:29.087316 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.087317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.087341 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.087424 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:09:29.087560 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.088035 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.088802 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.089004 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.089080 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.089272 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:09:29.089380 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:09:29.089482 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.089500 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.089522 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:09:29.091487 1682 raft_consensus.cc:1261] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Refusing update from remote peer 8e66ebc7ed4e4285821aeab116a13368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:29.091487 1229 raft_consensus.cc:1261] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Refusing update from remote peer 8e66ebc7ed4e4285821aeab116a13368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:29.091882 1826 consensus_queue.cc:1035] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:29.092061 1833 consensus_queue.cc:1035] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:29.095486 1842 mvcc.cc:205] Tried to move back new op lower bound from 7317476839795937280 to 7317476839755517952. Current Snapshot: MvccSnapshot[applied={T|T < 7317476839795937280 or (T in {7317476839795937280})}]
I20260812 00:09:29.097283 985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:29.097399 985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:29.098212 985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Sending DeleteTablet for 3 replicas of tablet bcf429a3e861461bba4fac835f305f24
I20260812 00:09:29.098589 1408 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf429a3e861461bba4fac835f305f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:29 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:36514
I20260812 00:09:29.098615 1644 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf429a3e861461bba4fac835f305f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:29 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:59044
I20260812 00:09:29.098871 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 1079
I20260812 00:09:29.098886 1872 tablet_replica.cc:330] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: stopping tablet replica
I20260812 00:09:29.098925 1196 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf429a3e861461bba4fac835f305f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:29 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:60612
I20260812 00:09:29.099050 1872 raft_consensus.cc:2229] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:09:29.107931 1873 tablet_replica.cc:330] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: stopping tablet replica
I20260812 00:09:29.108345 1873 raft_consensus.cc:2229] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:29.108589 1873 raft_consensus.cc:2258] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:29.110459 1873 ts_tablet_manager.cc:1905] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:29.119724 1873 ts_tablet_manager.cc:1918] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:29.099175 1872 raft_consensus.cc:2258] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:29.120097 1873 log.cc:1196] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/bcf429a3e861461bba4fac835f305f24
I20260812 00:09:29.120488 1873 ts_tablet_manager.cc:1939] T bcf429a3e861461bba4fac835f305f24 P 8d3a413bfa7f49ebaf90aff344964e83: Deleting consensus metadata
I20260812 00:09:29.121096 968 catalog_manager.cc:4841] TS 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): tablet bcf429a3e861461bba4fac835f305f24 (table test-table [id=91a37d93021e466fa0a666b6087fa124]) successfully deleted
I20260812 00:09:29.122004 1872 ts_tablet_manager.cc:1905] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:29.124038 1872 ts_tablet_manager.cc:1918] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:29.124215 1872 log.cc:1196] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/bcf429a3e861461bba4fac835f305f24
I20260812 00:09:29.124581 1872 ts_tablet_manager.cc:1939] T bcf429a3e861461bba4fac835f305f24 P 8e66ebc7ed4e4285821aeab116a13368: Deleting consensus metadata
I20260812 00:09:29.125201 967 catalog_manager.cc:4841] TS 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): tablet bcf429a3e861461bba4fac835f305f24 (table test-table [id=91a37d93021e466fa0a666b6087fa124]) successfully deleted
W20260812 00:09:29.127007 967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:41713 (error 108)
I20260812 00:09:29.127648 1800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.127501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:41713 (local address 127.0.0.1:60620)
0812 00:09:29.127564 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.127610 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:41713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 00:09:29.127727 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 1309
W20260812 00:09:29.142683 967 catalog_manager.cc:4584] TS de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bcf429a3e861461bba4fac835f305f24: Network error: Client connection negotiation failed: client connection to 127.7.85.193:41713: connect: Connection refused (error 111)
I20260812 00:09:29.155079 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 1534
I20260812 00:09:29.172303 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 931
I20260812 00:09:29.182566 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33245
--webserver_interface=127.7.85.254
--webserver_port=34953
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:29.272548 1876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:29.272819 1876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:29.272861 1876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:29.274111 1876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:29.274192 1876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:29.274219 1876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:29.274245 1876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:29.274277 1876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:29.275872 1876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=34953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:29.276188 1876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:29.276453 1876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:29.287614 1885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.287614 1890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.287633 1888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:29.288126 1876 server_base.cc:1034] running on GCE node
I20260812 00:09:29.288435 1876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:29.289017 1876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:29.290196 1876 hybrid_clock.cc:648] HybridClock initialized: now 1786493369290192 us; error 39 us; skew 500 ppm
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:29.293376 1876 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:29.293959 1876 webserver.cc:460] Webserver started at http://127.7.85.254:34953/ using document root <none> and password file <none>
I20260812 00:09:29.294243 1876 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:29.294353 1876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:29.296020 1876 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:29.296964 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.297264 1876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:29.297374 1876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "cb7707ba261c4082940d9ab37a866108"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "085d9da81389e631944d963d2220a190"
server_key_iv: "4df886f9ba670b18fb5c71f666909c91"
server_key_version: "encryptionkey@0"
I20260812 00:09:29.297875 1876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:29.312681 1876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:29.315888 1876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:29.316103 1876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:29.320961 1876 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33245
I20260812 00:09:29.321002 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33245 every 8 connection(s)
I20260812 00:09:29.321502 1876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:29.323096 2003 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:09:29.323781 2003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap starting.
I20260812 00:09:29.329913 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 1876
I20260812 00:09:29.330305 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:41713
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:33245
--webserver_port=43765
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:29.334501 2003 log.cc:824] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.336686 2003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:09:29.337054 2003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap complete.
I20260812 00:09:29.339953 2003 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:29.340233 2003 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb7707ba261c4082940d9ab37a866108, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.340420 2003 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:29.340505 2003 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:29.340580 2003 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:29.340634 2003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:09:29.341979 2003 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:29.342088 2003 leader_election.cc:302] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb7707ba261c4082940d9ab37a866108; no voters:
I20260812 00:09:29.342321 2003 leader_election.cc:288] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:09:29.342363 2012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:09:29.342482 2012 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 LEADER]: Becoming Leader. State: Replica: cb7707ba261c4082940d9ab37a866108, State: Running, Role: LEADER
I20260812 00:09:29.342578 2012 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:29.342631 2003 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:29.342900 2013 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cb7707ba261c4082940d9ab37a866108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } } }
I20260812 00:09:29.343000 2013 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:29.342895 2012 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb7707ba261c4082940d9ab37a866108. Latest consensus state: current_term: 2 leader_uuid: "cb7707ba261c4082940d9ab37a866108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } } }
I20260812 00:09:29.343223 2012 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:29.343525 2018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:29.344517 2018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:29.344625 2018 catalog_manager.cc:1259] Loaded cluster ID: 12e95ae5f512491a8f8a226900523bf7
I20260812 00:09:29.344667 2018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:09:29.345093 2018 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 00:09:29.347941 2018 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 00:09:29.350590 2018 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:29.350878 2018 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Loaded TSK: 0
I20260812 00:09:29.351125 2018 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:09:29.439139 2009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:29.439392 2009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:29.439419 2009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:29.440735 2009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:29.440791 2009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:29.440821 2009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:09:29.442487 2009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:41713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=43765
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:29.442785 2009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:29.443063 2009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:29.449860 2052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.449983 2057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.449961 2061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:29.450341 2009 server_base.cc:1034] running on GCE node
I20260812 00:09:29.450603 2009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:29.450881 2009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:29.452080 2009 hybrid_clock.cc:648] HybridClock initialized: now 1786493369452061 us; error 33 us; skew 500 ppm
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:29.455420 2009 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:09:29.456018 2009 webserver.cc:460] Webserver started at http://127.7.85.193:43765/ using document root <none> and password file <none>
I20260812 00:09:29.456280 2009 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:29.456370 2009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:29.458003 2009 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:09:29.458779 2075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.459038 2009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:09:29.459156 2009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "de9e8fbebb704a8293f078551f000192"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "59521535a7c6b46c68305d9c21ebd702"
server_key_iv: "c2481898e32b34ba8cac1deca6378642"
server_key_version: "encryptionkey@0"
I20260812 00:09:29.459642 2009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:29.479486 2009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:29.483350 2009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:29.483532 2009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:29.483875 2009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:29.484422 2085 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 00:09:29.486110 2009 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 00:09:29.486173 2009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 00:09:29.486205 2009 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 00:09:29.486955 2009 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 00:09:29.487011 2009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 00:09:29.487052 2085 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Bootstrap starting.
I20260812 00:09:29.497560 2009 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:41713
I20260812 00:09:29.497738 2278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:41713 every 8 connection(s)
I20260812 00:09:29.498142 2009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:09:29.498451 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 2009
I20260812 00:09:29.498770 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:35467
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:33245
--webserver_port=39803
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:29.502483 2085 log.cc:824] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.505887 2085 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:09:29.506286 2085 tablet_bootstrap.cc:492] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Bootstrap complete.
I20260812 00:09:29.507859 2085 ts_tablet_manager.cc:1397] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.007s
I20260812 00:09:29.509594 2085 raft_consensus.cc:348] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.510432 2085 raft_consensus.cc:729] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9e8fbebb704a8293f078551f000192, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.510586 2085 consensus_queue.cc:260] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } } peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } }
I20260812 00:09:29.510922 2085 ts_tablet_manager.cc:1428] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Time spent starting tablet: real 0.003s user 0.002s sys 0.003s
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:29.517556 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.499779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:45265 (local address 127.7.85.254:33245)
0812 00:09:29.503036 (+ 3257us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.503046 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.503101 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:09:29.503453 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.503460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.503521 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.503648 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:29.504680 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.506415 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.508279 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.508962 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.509060 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.512748 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:29.512756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:29.512757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:29.512773 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:29.515311 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.515898 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.515903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.515903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.516042 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.516351 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.516352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.516353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.516672 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:29.516761 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.517152 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.517342 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3197,"thread_start_us":117,"threads_started":1}
I20260812 00:09:29.517560 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.499332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:45265)
0812 00:09:29.501041 (+ 1709us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.501100 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.503181 (+ 2081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.503797 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.503802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.504244 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:29.504530 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.504538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.506580 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.506581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.508139 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.508144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.509312 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.509314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.509473 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.510223 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:29.510234 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:29.512592 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:29.515443 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.515455 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.515458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.515792 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.516130 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.516130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.516131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.516277 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.516759 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:29.516767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:29.516945 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.517067 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.517322 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":124,"threads_started":1}
I20260812 00:09:29.518707 2283 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33245
I20260812 00:09:29.518854 2283 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:29.519063 2283 heartbeater.cc:511] Master 127.7.85.254:33245 requested a full tablet report, sending...
I20260812 00:09:29.519912 1943 ts_manager.cc:194] Registered new tserver with Master: de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713)
I20260812 00:09:29.521096 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:45265
I20260812 00:09:29.522210 2283 heartbeater.cc:503] Master 127.7.85.254:33245 was elected leader, sending a full tablet report...
I20260812 00:09:29.529047 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.520792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:41713 (local address 127.0.0.1:60634)
0812 00:09:29.521022 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.521284 (+ 262us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.521362 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.525555 (+ 4193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.525560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.525564 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.525657 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.525660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.526667 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.526668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.527961 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.527965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.528698 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.528699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.528792 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.528808 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.528888 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.528950 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":133,"threads_started":1}
I20260812 00:09:29.529304 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.521103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.7.85.193:41713)
0812 00:09:29.525353 (+ 4250us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.525359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.525372 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:09:29.525407 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.525409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.525425 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.525511 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.525856 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.526574 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.528085 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.528596 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.528663 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.528711 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.528980 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.529213 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4208,"thread_start_us":119,"threads_started":1}
I20260812 00:09:29.529839 2163 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf429a3e861461bba4fac835f305f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:29 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:60634
I20260812 00:09:29.530592 2294 tablet_replica.cc:330] stopping tablet replica
I20260812 00:09:29.530879 2294 raft_consensus.cc:2229] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:29.530985 2294 raft_consensus.cc:2258] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:29.531363 2294 ts_tablet_manager.cc:1905] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:29.533499 2294 ts_tablet_manager.cc:1918] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:29.533609 2294 log.cc:1196] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/bcf429a3e861461bba4fac835f305f24
I20260812 00:09:29.534050 2294 ts_tablet_manager.cc:1939] T bcf429a3e861461bba4fac835f305f24 P de9e8fbebb704a8293f078551f000192: Deleting consensus metadata
I20260812 00:09:29.534713 1913 catalog_manager.cc:4841] TS de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713): tablet bcf429a3e861461bba4fac835f305f24 (table test-table [id=91a37d93021e466fa0a666b6087fa124]) successfully deleted
W20260812 00:09:29.601269 2288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:29.601518 2288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:29.601572 2288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:29.602958 2288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:29.603014 2288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:29.603044 2288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:09:29.604718 2288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:35467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=39803
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:29.605007 2288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:29.605314 2288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:29.610742 2307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.610931 2302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.610841 2303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:29.611269 2288 server_base.cc:1034] running on GCE node
I20260812 00:09:29.611528 2288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:29.611809 2288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:29.613004 2288 hybrid_clock.cc:648] HybridClock initialized: now 1786493369612993 us; error 32 us; skew 500 ppm
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:29.615938 2288 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:09:29.616381 2288 webserver.cc:460] Webserver started at http://127.7.85.194:39803/ using document root <none> and password file <none>
I20260812 00:09:29.616596 2288 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:29.616654 2288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:29.618153 2288 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:29.618945 2321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.619262 2288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:29.619362 2288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "8e66ebc7ed4e4285821aeab116a13368"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "09938fac72808034e8c81a01243d682b"
server_key_iv: "7492aa16e713e5f083b12abd22cf6d45"
server_key_version: "encryptionkey@0"
I20260812 00:09:29.619729 2288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:29.636183 2288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:29.638863 2288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:29.638989 2288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:29.639202 2288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:29.639547 2288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:29.639585 2288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.639611 2288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:29.639657 2288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.647176 2288 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35467
I20260812 00:09:29.647370 2526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35467 every 8 connection(s)
I20260812 00:09:29.647693 2288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:09:29.648221 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 2288
I20260812 00:09:29.648496 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:34141
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:33245
--webserver_port=41607
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:29.660463 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.648607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:58955 (local address 127.7.85.254:33245)
0812 00:09:29.648665 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.648667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.648989 (+ 322us) server_negotiation.cc:407] Connection header received
0812 00:09:29.649866 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.649867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.649878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.649931 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:29.651136 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.651649 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.653299 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.653524 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.653596 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.656720 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:29.656722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:29.656723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:29.656733 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:29.658778 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.659244 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.659245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.659246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.659328 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.659709 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.659711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.659711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.659867 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:29.659949 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.660249 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.660360 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:09:29.660686 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.648621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:58955)
0812 00:09:29.648827 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.648881 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.649737 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.650004 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.650008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.650806 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:29.651016 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.651021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.651760 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.651762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.653091 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.653096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.653637 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.653639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.653808 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.654444 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:29.654456 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:29.656604 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:29.658888 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.658897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.658900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.659158 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.659450 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.659452 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.659452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.659596 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.659940 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:29.659948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:29.660101 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.660225 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.660423 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260812 00:09:29.661279 2527 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33245
I20260812 00:09:29.661386 2527 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:29.661609 2527 heartbeater.cc:511] Master 127.7.85.254:33245 requested a full tablet report, sending...
I20260812 00:09:29.662110 1943 ts_manager.cc:194] Registered new tserver with Master: 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
I20260812 00:09:29.662511 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:58955
W20260812 00:09:29.744246 2535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:29.744481 2535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:29.744508 2535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:29.745801 2535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:29.745848 2535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:29.745878 2535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:09:29.747452 2535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:34141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=41607
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:29.747721 2535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:29.747982 2535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:29.753616 2548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.753615 2545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:29.753615 2553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:29.753692 2535 server_base.cc:1034] running on GCE node
I20260812 00:09:29.754175 2535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:29.754480 2535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:29.755654 2535 hybrid_clock.cc:648] HybridClock initialized: now 1786493369755632 us; error 41 us; skew 500 ppm
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:29.758685 2535 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:09:29.759176 2535 webserver.cc:460] Webserver started at http://127.7.85.195:41607/ using document root <none> and password file <none>
I20260812 00:09:29.759421 2535 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:29.759500 2535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:29.761024 2535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:09:29.761827 2567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.762298 2535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:29.762415 2535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "8d3a413bfa7f49ebaf90aff344964e83"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "bc3c0e130a4c2e091787ed2c21e46f07"
server_key_iv: "392bf141e97c742d3162b85007e466cd"
server_key_version: "encryptionkey@0"
I20260812 00:09:29.762830 2535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:29.772410 2535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:29.775559 2535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:29.775767 2535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:29.776065 2535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:29.776432 2535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:29.776494 2535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.776538 2535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:29.776567 2535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.784067 2535 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34141
I20260812 00:09:29.784154 2761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34141 every 8 connection(s)
I20260812 00:09:29.784636 2535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:09:29.786592 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 2535
Aug 12 00:09:29 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:29.796219 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.787152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.7.85.254:33245)
0812 00:09:29.787346 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.787348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.787382 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:09:29.787452 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.787453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.787463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.787525 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:29.788407 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.789135 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.790312 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.790515 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.790580 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.792593 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:29.792595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:29.792595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:29.792603 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:29.794862 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.795278 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.795279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.795280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.795347 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.795569 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.795570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.795570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.795700 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:29.795929 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.795969 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.796101 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
I20260812 00:09:29.798122 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.785972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:57197 (local address 127.7.85.254:33245)
0812 00:09:29.786091 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.786093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.786621 (+ 528us) server_negotiation.cc:407] Connection header received
0812 00:09:29.787347 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.787348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.787361 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.787423 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:29.788748 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.789414 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.790805 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.791108 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.791187 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.794545 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:29.794547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:29.794547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:29.794554 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:29.796264 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.796861 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.796862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.796862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.796950 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:29.797396 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:29.797397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:29.797397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:29.797537 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:29.797613 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.797969 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.798047 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 00:09:29.798350 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.785957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:57197)
0812 00:09:29.786383 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.786445 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.787217 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.787524 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.787528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.788409 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:29.788636 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.788642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.789504 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.789505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.790691 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.790694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.791295 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.791298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.791540 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.792212 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:29.792226 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:29.794411 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:29.796365 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.796376 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.796379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.796784 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.797056 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:29.797057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:29.797057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:29.797338 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:29.797645 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:29.797653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:29.797795 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.797907 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.798069 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":157,"threads_started":1}
I20260812 00:09:29.798882 2770 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33245
I20260812 00:09:29.798983 2770 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:29.799135 2770 heartbeater.cc:511] Master 127.7.85.254:33245 requested a full tablet report, sending...
I20260812 00:09:29.799528 1943 ts_manager.cc:194] Registered new tserver with Master: 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141)
I20260812 00:09:29.799880 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:57197
I20260812 00:09:29.809142 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:09:29.812656 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.809864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.7.85.254:33245)
0812 00:09:29.809943 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.809946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.810082 (+ 136us) server_negotiation.cc:407] Connection header received
0812 00:09:29.810197 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.810198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.810240 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.810328 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:09:29.810484 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.810993 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.811800 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.811994 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.812109 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.812290 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:09:29.812411 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:09:29.812515 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.812533 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.812554 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:09:29.812974 1943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:09:29.815150 1943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:09:29.818084 2163 tablet_service.cc:1467] Processing CreateTablet for tablet 3794414e0582468994ab7998c4302a3b (DEFAULT_TABLE table=test-table [id=86e6b088ada7489681032e494bc5a239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:29.818351 2163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3794414e0582468994ab7998c4302a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:29.820106 2085 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Bootstrap starting.
I20260812 00:09:29.820487 2085 tablet_bootstrap.cc:654] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:29.821084 2085 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: No bootstrap required, opened a new log
I20260812 00:09:29.821143 2085 ts_tablet_manager.cc:1397] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 00:09:29.821298 2085 raft_consensus.cc:348] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:29.821378 2085 raft_consensus.cc:374] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:29.821424 2085 raft_consensus.cc:729] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9e8fbebb704a8293f078551f000192, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.821483 2085 consensus_queue.cc:260] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:29.821574 2085 ts_tablet_manager.cc:1428] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:29.822484 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.817910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.7.85.194:35467)
0812 00:09:29.818117 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.818123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.818134 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:09:29.818177 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.818179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.818196 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.818280 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.818745 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.819830 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.821171 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.821997 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.822102 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.822295 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.822340 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.822383 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260812 00:09:29.822482 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.817468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35467 (local address 127.0.0.1:36528)
0812 00:09:29.817735 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.817748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.817804 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.818460 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.818461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.818463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.818516 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.818518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.819939 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.819940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.821060 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.821064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.822103 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.822104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.822170 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.822174 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.822327 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.822383 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":65,"threads_started":1}
I20260812 00:09:29.822499 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.817432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34141 (local address 127.0.0.1:59056)
0812 00:09:29.817592 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:29.817606 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:29.817656 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:29.818394 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:29.818395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:29.818397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:29.818447 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.818449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.819911 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.819913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.820927 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:29.820930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.822214 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.822215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:29.822283 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.822287 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:29.822317 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 00:09:29.822367 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260812 00:09:29.822897 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:29.817551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.7.85.195:34141)
0812 00:09:29.818107 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:29.818111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:29.818120 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:09:29.818155 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:29.818157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:29.818170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:29.818349 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:29.818613 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.819785 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.821034 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:29.822115 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:29.822452 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:29.822518 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:29.822547 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 00:09:29.822654 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":114,"threads_started":1}
I20260812 00:09:29.823712 2412 tablet_service.cc:1467] Processing CreateTablet for tablet 3794414e0582468994ab7998c4302a3b (DEFAULT_TABLE table=test-table [id=86e6b088ada7489681032e494bc5a239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:29.823712 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 3794414e0582468994ab7998c4302a3b (DEFAULT_TABLE table=test-table [id=86e6b088ada7489681032e494bc5a239]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:29.824117 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3794414e0582468994ab7998c4302a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:29.824117 2412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3794414e0582468994ab7998c4302a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:29.827289 2784 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Bootstrap starting.
I20260812 00:09:29.827430 2788 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Bootstrap starting.
I20260812 00:09:29.827984 2784 tablet_bootstrap.cc:654] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:29.828001 2788 tablet_bootstrap.cc:654] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:29.828503 2788 log.cc:824] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.828517 2784 log.cc:824] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:29.829351 2788 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: No bootstrap required, opened a new log
I20260812 00:09:29.829377 2784 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: No bootstrap required, opened a new log
I20260812 00:09:29.829437 2788 ts_tablet_manager.cc:1397] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:09:29.829499 2784 ts_tablet_manager.cc:1397] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:09:29.831193 2788 raft_consensus.cc:348] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:29.831189 2784 raft_consensus.cc:348] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:29.831377 2788 raft_consensus.cc:374] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:29.831374 2784 raft_consensus.cc:374] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:29.831426 2784 raft_consensus.cc:729] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d3a413bfa7f49ebaf90aff344964e83, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.831426 2788 raft_consensus.cc:729] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e66ebc7ed4e4285821aeab116a13368, State: Initialized, Role: FOLLOWER
I20260812 00:09:29.831560 2788 consensus_queue.cc:260] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:29.831557 2784 consensus_queue.cc:260] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:29.831887 2784 ts_tablet_manager.cc:1428] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 00:09:29.831892 2527 heartbeater.cc:503] Master 127.7.85.254:33245 was elected leader, sending a full tablet report...
I20260812 00:09:29.831892 2770 heartbeater.cc:503] Master 127.7.85.254:33245 was elected leader, sending a full tablet report...
I20260812 00:09:29.831885 2788 ts_tablet_manager.cc:1428] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 00:09:29.898451 2528 tablet.cc:2369] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:09:29.999210 2284 tablet.cc:2369] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:09:30.035425 2771 tablet.cc:2369] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:09:30.055289 2798 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:30.055490 2798 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:30.055902 2798 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467), de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713)
I20260812 00:09:30.059285 2801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.056483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:42137 (local address 127.7.85.193:41713)
0812 00:09:30.056607 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:30.056609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:30.056617 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:09:30.056701 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:30.056702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:30.056711 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:30.056793 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:30.056970 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.057535 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.058490 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.058885 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.058980 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:30.058991 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:30.059146 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 00:09:30.059193 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260812 00:09:30.059296 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.056396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:41713 (local address 127.7.85.195:42137)
0812 00:09:30.056549 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.056557 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:30.056626 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:30.056802 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:30.056804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:30.056806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:30.056890 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:30.056892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.057627 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.057628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:30.058384 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:30.058385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.058979 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.058980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:30.059064 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:30.059068 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:30.059139 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 00:09:30.059193 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260812 00:09:30.059319 2782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.056417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:48051 (local address 127.7.85.194:35467)
0812 00:09:30.056472 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:30.056474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:30.056485 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:09:30.056521 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:30.056522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:30.056537 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:30.056627 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:30.056840 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.057410 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.058409 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.058828 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.058996 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:30.059010 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:30.059190 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 00:09:30.059239 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:09:30.059393 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.056238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35467 (local address 127.7.85.195:48051)
0812 00:09:30.056336 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.056351 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:30.056407 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:30.056643 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:30.056644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:30.056647 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:30.056768 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:30.056771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.057502 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.057504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:30.058300 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:30.058302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.058999 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.059000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:30.059100 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:30.059103 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:09:30.059166 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 00:09:30.059254 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 00:09:30.059819 2194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3794414e0582468994ab7998c4302a3b" candidate_uuid: "8d3a413bfa7f49ebaf90aff344964e83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9e8fbebb704a8293f078551f000192" is_pre_election: true
I20260812 00:09:30.059943 2446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3794414e0582468994ab7998c4302a3b" candidate_uuid: "8d3a413bfa7f49ebaf90aff344964e83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e66ebc7ed4e4285821aeab116a13368" is_pre_election: true
I20260812 00:09:30.060015 2194 raft_consensus.cc:2454] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d3a413bfa7f49ebaf90aff344964e83 in term 0.
I20260812 00:09:30.060120 2446 raft_consensus.cc:2454] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d3a413bfa7f49ebaf90aff344964e83 in term 0.
I20260812 00:09:30.060294 2570 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d3a413bfa7f49ebaf90aff344964e83, de9e8fbebb704a8293f078551f000192; no voters:
I20260812 00:09:30.060436 2798 raft_consensus.cc:2781] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:09:30.060496 2798 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:09:30.060525 2798 raft_consensus.cc:3037] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:30.061331 2798 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:30.061678 2798 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [CANDIDATE]: Term 1 election: Requested vote from peers 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467), de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713)
I20260812 00:09:30.061954 2446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3794414e0582468994ab7998c4302a3b" candidate_uuid: "8d3a413bfa7f49ebaf90aff344964e83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e66ebc7ed4e4285821aeab116a13368"
I20260812 00:09:30.061954 2194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3794414e0582468994ab7998c4302a3b" candidate_uuid: "8d3a413bfa7f49ebaf90aff344964e83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9e8fbebb704a8293f078551f000192"
I20260812 00:09:30.062077 2194 raft_consensus.cc:3037] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:30.062093 2446 raft_consensus.cc:3037] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:30.063010 2194 raft_consensus.cc:2454] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d3a413bfa7f49ebaf90aff344964e83 in term 1.
I20260812 00:09:30.063045 2446 raft_consensus.cc:2454] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d3a413bfa7f49ebaf90aff344964e83 in term 1.
I20260812 00:09:30.063225 2570 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d3a413bfa7f49ebaf90aff344964e83, de9e8fbebb704a8293f078551f000192; no voters:
I20260812 00:09:30.063339 2798 raft_consensus.cc:2781] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:30.063536 2798 raft_consensus.cc:686] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 LEADER]: Becoming Leader. State: Replica: 8d3a413bfa7f49ebaf90aff344964e83, State: Running, Role: LEADER
I20260812 00:09:30.063652 2798 consensus_queue.cc:237] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:30.064620 1943 catalog_manager.cc:5495] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "8d3a413bfa7f49ebaf90aff344964e83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } health_report { overall_health: UNKNOWN } } }
I20260812 00:09:30.110284 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.106916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.7.85.195:34141)
0812 00:09:30.106981 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:30.106983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:30.106992 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:09:30.107059 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:30.107060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:30.107082 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:30.107139 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:09:30.107363 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.107854 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.108619 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:30.108843 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:30.108911 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:30.109105 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:09:30.109482 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:09:30.109581 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:30.109915 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 00:09:30.110005 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 00:09:30.113328 2446 raft_consensus.cc:1261] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Refusing update from remote peer 8d3a413bfa7f49ebaf90aff344964e83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:30.113703 2194 raft_consensus.cc:1261] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Refusing update from remote peer 8d3a413bfa7f49ebaf90aff344964e83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:30.113794 2798 consensus_queue.cc:1035] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:30.114108 2804 consensus_queue.cc:1035] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:30.117013 2814 mvcc.cc:205] Tried to move back new op lower bound from 7317476843979800576 to 7317476843781537792. Current Snapshot: MvccSnapshot[applied={T|T < 7317476843979800576}]
I20260812 00:09:30.117318 2815 mvcc.cc:205] Tried to move back new op lower bound from 7317476843979800576 to 7317476843781537792. Current Snapshot: MvccSnapshot[applied={T|T < 7317476843979800576}]
I20260812 00:09:30.119391 1943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:30.119513 1943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:30.120472 1943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Sending DeleteTablet for 3 replicas of tablet 3794414e0582468994ab7998c4302a3b
I20260812 00:09:30.120878 2412 tablet_service.cc:1514] Processing DeleteTablet for tablet 3794414e0582468994ab7998c4302a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:30 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:36528
I20260812 00:09:30.120918 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 3794414e0582468994ab7998c4302a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:30 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:59056
I20260812 00:09:30.121155 2163 tablet_service.cc:1514] Processing DeleteTablet for tablet 3794414e0582468994ab7998c4302a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:30 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:60634
I20260812 00:09:30.121263 2829 tablet_replica.cc:330] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: stopping tablet replica
I20260812 00:09:30.121366 2828 tablet_replica.cc:330] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: stopping tablet replica
I20260812 00:09:30.121500 2833 tablet_replica.cc:330] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: stopping tablet replica
I20260812 00:09:30.121505 2829 raft_consensus.cc:2229] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:09:30.121552 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 2009
I20260812 00:09:30.121632 2833 raft_consensus.cc:2229] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:30.138711 2828 raft_consensus.cc:2229] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:30.138926 2828 raft_consensus.cc:2258] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:30.121622 2829 raft_consensus.cc:2258] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:30.144719 2829 ts_tablet_manager.cc:1905] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:30.147315 2829 ts_tablet_manager.cc:1918] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:30.147423 2829 log.cc:1196] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/3794414e0582468994ab7998c4302a3b
I20260812 00:09:30.147781 2829 ts_tablet_manager.cc:1939] T 3794414e0582468994ab7998c4302a3b P 8d3a413bfa7f49ebaf90aff344964e83: Deleting consensus metadata
I20260812 00:09:30.148330 1914 catalog_manager.cc:4841] TS 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): tablet 3794414e0582468994ab7998c4302a3b (table test-table [id=86e6b088ada7489681032e494bc5a239]) successfully deleted
I20260812 00:09:30.154060 2828 ts_tablet_manager.cc:1905] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:30.157603 2828 ts_tablet_manager.cc:1918] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:30.157730 2828 log.cc:1196] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/3794414e0582468994ab7998c4302a3b
I20260812 00:09:30.158028 2828 ts_tablet_manager.cc:1939] T 3794414e0582468994ab7998c4302a3b P 8e66ebc7ed4e4285821aeab116a13368: Deleting consensus metadata
I20260812 00:09:30.158627 1913 catalog_manager.cc:4841] TS 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): tablet 3794414e0582468994ab7998c4302a3b (table test-table [id=86e6b088ada7489681032e494bc5a239]) successfully deleted
W20260812 00:09:30.160058 1913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:41713 (error 108)
I20260812 00:09:30.160409 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 2288
I20260812 00:09:30.161674 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.160802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:41713 (local address 127.0.0.1:60642)
0812 00:09:30.161537 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.161609 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:41713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":684}
W20260812 00:09:30.169293 1913 catalog_manager.cc:4584] TS de9e8fbebb704a8293f078551f000192 (127.7.85.193:41713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3794414e0582468994ab7998c4302a3b: Network error: Client connection negotiation failed: client connection to 127.7.85.193:41713: connect: Connection refused (error 111)
I20260812 00:09:30.183724 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 2535
I20260812 00:09:30.205677 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 1876
I20260812 00:09:30.216218 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33245
--webserver_interface=127.7.85.254
--webserver_port=34953
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:30.306768 2840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:30.307006 2840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:30.307031 2840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:30.308152 2840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:30.308192 2840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:30.308207 2840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:30.308219 2840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:30.308233 2840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:30.309824 2840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=34953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:30.310051 2840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:30.310246 2840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:30.315559 2855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.315551 2848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.315553 2849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:30.315582 2840 server_base.cc:1034] running on GCE node
I20260812 00:09:30.316066 2840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:30.316385 2840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:30.317567 2840 hybrid_clock.cc:648] HybridClock initialized: now 1786493370317536 us; error 53 us; skew 500 ppm
Aug 12 00:09:30 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:30.321645 2840 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:30.324050 2840 webserver.cc:460] Webserver started at http://127.7.85.254:34953/ using document root <none> and password file <none>
I20260812 00:09:30.324342 2840 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:30.324395 2840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:30.325989 2840 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:30.326864 2868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.327085 2840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:30.327199 2840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "cb7707ba261c4082940d9ab37a866108"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "085d9da81389e631944d963d2220a190"
server_key_iv: "4df886f9ba670b18fb5c71f666909c91"
server_key_version: "encryptionkey@0"
I20260812 00:09:30.327663 2840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:30.340353 2840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:30.342974 2840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:30.343106 2840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:30.347312 2840 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33245
I20260812 00:09:30.347412 2964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33245 every 8 connection(s)
I20260812 00:09:30.347805 2840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:30.349712 2966 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:09:30.350364 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap starting.
I20260812 00:09:30.354259 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 2840
I20260812 00:09:30.354578 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:41713
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:33245
--webserver_port=43765
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:30.360303 2966 log.cc:824] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:30.362967 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:09:30.363339 2966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: Bootstrap complete.
I20260812 00:09:30.366199 2966 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:30.366485 2966 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb7707ba261c4082940d9ab37a866108, State: Initialized, Role: FOLLOWER
I20260812 00:09:30.366677 2966 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:30.366763 2966 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:30.366796 2966 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:30.366873 2966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:09:30.367964 2966 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:30.368062 2966 leader_election.cc:302] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb7707ba261c4082940d9ab37a866108; no voters:
I20260812 00:09:30.368314 2966 leader_election.cc:288] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 00:09:30.368355 2973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 00:09:30.368512 2973 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [term 3 LEADER]: Becoming Leader. State: Replica: cb7707ba261c4082940d9ab37a866108, State: Running, Role: LEADER
I20260812 00:09:30.368666 2973 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } }
I20260812 00:09:30.368673 2966 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:30.369091 2975 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cb7707ba261c4082940d9ab37a866108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } } }
I20260812 00:09:30.369199 2975 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:30.369380 2977 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb7707ba261c4082940d9ab37a866108. Latest consensus state: current_term: 3 leader_uuid: "cb7707ba261c4082940d9ab37a866108" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb7707ba261c4082940d9ab37a866108" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33245 } } }
I20260812 00:09:30.369459 2977 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:30.370898 3000 catalog_manager.cc:1259] Loaded cluster ID: 12e95ae5f512491a8f8a226900523bf7
I20260812 00:09:30.370954 3000 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: loading cluster ID for follower catalog manager: success
W20260812 00:09:30.371263 3000 catalog_manager.cc:1576] T 00000000000000000000000000000000 P cb7707ba261c4082940d9ab37a866108: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260812 00:09:30.371315 3000 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260812 00:09:30.371357 2980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:30.371773 2980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:30.371847 2980 catalog_manager.cc:1259] Loaded cluster ID: 12e95ae5f512491a8f8a226900523bf7
I20260812 00:09:30.371872 2980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 00:09:30.371994 2980 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f487faa49ad google::LogMessage::Fail() at ??:0
@ 0x7f487faa788a google::LogMessage::SendToLog() at ??:0
@ 0x7f487faa44de google::LogMessage::Flush() at ??:0
@ 0x7f487faa5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f48879c90f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f48879d28b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f48819720d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4881967954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f48827cb6db start_thread at ??:0
@ 0x7f487e5a261f clone at ??:0
W20260812 00:09:30.457800 2968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:30.458069 2968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:30.458112 2968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:30.459331 2968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:30.459409 2968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:30.459457 2968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:09:30.461090 2968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:41713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=43765
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:30.461453 2968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:30.461716 2968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:09:30.467972 2968 server_base.cc:1034] running on GCE node
W20260812 00:09:30.467938 3026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.467940 3022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.468222 3021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:30.468494 2968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:30.468791 2968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:30.469949 2968 hybrid_clock.cc:648] HybridClock initialized: now 1786493370469932 us; error 34 us; skew 500 ppm
Aug 12 00:09:30 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:30.472759 2968 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:09:30.473519 2968 webserver.cc:460] Webserver started at http://127.7.85.193:43765/ using document root <none> and password file <none>
I20260812 00:09:30.473788 2968 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:30.473860 2968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:30.475399 2968 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:30.476363 3040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.476641 2968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:30.476760 2968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "de9e8fbebb704a8293f078551f000192"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "59521535a7c6b46c68305d9c21ebd702"
server_key_iv: "c2481898e32b34ba8cac1deca6378642"
server_key_version: "encryptionkey@0"
I20260812 00:09:30.477337 2968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:30.504638 2968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:30.507292 2968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:30.507491 2968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:30.507763 2968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:30.508284 3054 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 00:09:30.510437 2968 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 00:09:30.510532 2968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 00:09:30.510588 2968 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 00:09:30.511291 2968 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 00:09:30.511353 2968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 00:09:30.511435 3054 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Bootstrap starting.
I20260812 00:09:30.519308 2968 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:41713
I20260812 00:09:30.519488 3246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:41713 every 8 connection(s)
I20260812 00:09:30.519788 2968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:09:30.523751 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 2968
I20260812 00:09:30.523836 3054 log.cc:824] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:30.524185 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:35467
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:33245
--webserver_port=39803
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:09:30.527056 3054 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:09:30.527486 3054 tablet_bootstrap.cc:492] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Bootstrap complete.
I20260812 00:09:30.528976 3054 ts_tablet_manager.cc:1397] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.006s
I20260812 00:09:30.530567 3054 raft_consensus.cc:348] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:30.530944 3054 raft_consensus.cc:729] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9e8fbebb704a8293f078551f000192, State: Initialized, Role: FOLLOWER
I20260812 00:09:30.531138 3054 consensus_queue.cc:260] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:30.531458 3054 ts_tablet_manager.cc:1428] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260812 00:09:30.592765 3253 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:09:30.520631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:33817)
0812 00:09:30.520826 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.520872 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:30.521538 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:30.592504 (+ 70966us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
W20260812 00:09:30.593102 3249 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:33245 (0 consecutive failures): Network error: Failed to ping master at 127.7.85.254:33245: Client connection negotiation failed: client connection to 127.7.85.254:33245: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 00:09:30.593683 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.593422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:33255)
0812 00:09:30.593571 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.593626 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 00:09:30.594561 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.594369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:39725)
0812 00:09:30.594499 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.594535 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260812 00:09:30.594921 3249 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:09:30.616398 3260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:30.616674 3260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:30.616719 3260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:30.618126 3260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:30.618207 3260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:30.618258 3260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:09:30.619920 3260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:35467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=39803
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:30.620201 3260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:30.620455 3260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:30.625902 3276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.625917 3284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.625917 3279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:30.626210 3260 server_base.cc:1034] running on GCE node
I20260812 00:09:30.626466 3260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:30.626724 3260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:30.627907 3260 hybrid_clock.cc:648] HybridClock initialized: now 1786493370627890 us; error 33 us; skew 500 ppm
Aug 12 00:09:30 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:30.630674 3260 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:09:30.631114 3260 webserver.cc:460] Webserver started at http://127.7.85.194:39803/ using document root <none> and password file <none>
I20260812 00:09:30.631352 3260 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:30.631402 3260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:30.632815 3260 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:09:30.633558 3296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.633807 3260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:30.633920 3260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "8e66ebc7ed4e4285821aeab116a13368"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "09938fac72808034e8c81a01243d682b"
server_key_iv: "7492aa16e713e5f083b12abd22cf6d45"
server_key_version: "encryptionkey@0"
I20260812 00:09:30.634298 3260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:30.654228 3260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:30.656766 3260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:30.656890 3260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:30.657326 3260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:30.657692 3260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:30.657752 3260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.657799 3260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:30.657828 3260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.666004 3260 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35467
I20260812 00:09:30.666113 3497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35467 every 8 connection(s)
I20260812 00:09:30.666553 3260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:09:30.667869 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.667391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:48715)
0812 00:09:30.667572 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.667657 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
W20260812 00:09:30.669442 3498 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:33245 (0 consecutive failures): Network error: Failed to ping master at 127.7.85.254:33245: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
I20260812 00:09:30.670092 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.669720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:44533)
0812 00:09:30.670018 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.670065 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276}
I20260812 00:09:30.670915 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.670384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:55613)
0812 00:09:30.670737 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.670823 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297}
W20260812 00:09:30.671149 3498 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:09:30.671416 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 3260
I20260812 00:09:30.671833 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:34141
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:33245
--webserver_port=41607
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:30.762468 3504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:30.762701 3504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:30.762727 3504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:30.763912 3504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:30.763962 3504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:30.763990 3504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:09:30.765640 3504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:34141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=41607
--tserver_master_addrs=127.7.85.254:33245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:30.765933 3504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:30.766216 3504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:30.771713 3524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.771713 3517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:30.772011 3521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:30.772089 3504 server_base.cc:1034] running on GCE node
I20260812 00:09:30.772604 3504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:30.772835 3504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:30.773993 3504 hybrid_clock.cc:648] HybridClock initialized: now 1786493370773981 us; error 29 us; skew 500 ppm
Aug 12 00:09:30 dist-test-slave-z8x9 krb5kdc[908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:30.776710 3504 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:09:30.777328 3504 webserver.cc:460] Webserver started at http://127.7.85.195:41607/ using document root <none> and password file <none>
I20260812 00:09:30.777581 3504 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:30.777658 3504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:30.779066 3504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:30.779798 3539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.779989 3504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:30.780098 3504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "8d3a413bfa7f49ebaf90aff344964e83"
format_stamp: "Formatted at 2026-08-12 00:09:28 on dist-test-slave-z8x9"
server_key: "bc3c0e130a4c2e091787ed2c21e46f07"
server_key_iv: "392bf141e97c742d3162b85007e466cd"
server_key_version: "encryptionkey@0"
I20260812 00:09:30.780524 3504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:30.800006 3504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:30.802588 3504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:30.802775 3504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:30.803040 3504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:30.803393 3504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:30.803450 3504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.803498 3504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:30.803521 3504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:30.809803 3504 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34141
I20260812 00:09:30.809849 3734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34141 every 8 connection(s)
I20260812 00:09:30.810213 3504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:09:30.811350 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.810916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:40527)
0812 00:09:30.811103 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.811187 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
W20260812 00:09:30.811834 3735 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:33245 (0 consecutive failures): Network error: Failed to ping master at 127.7.85.254:33245: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
I20260812 00:09:30.812141 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.812038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:34507)
0812 00:09:30.812091 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.812120 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 00:09:30.812448 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:30.812369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:52573)
0812 00:09:30.812408 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:30.812432 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 00:09:30.812572 3735 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:09:30.819631 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 3504
I20260812 00:09:31.595819 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:31.595373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:35431)
0812 00:09:31.595589 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:31.595632 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":65,"threads_started":1}
I20260812 00:09:31.671764 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:31.671507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:40559)
0812 00:09:31.671655 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:31.671693 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260812 00:09:31.813270 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:31.812950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:37333)
0812 00:09:31.813093 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:31.813135 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260812 00:09:32.317471 3744 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:32.317615 3744 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:32.317981 3744 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141), 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
Aug 12 00:09:32 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.194@KRBTEST.COM
Aug 12 00:09:32 dist-test-slave-z8x9 krb5kdc[908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.195@KRBTEST.COM
I20260812 00:09:32.332718 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.318342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:33449 (local address 127.7.85.195:34141)
0812 00:09:32.318472 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:32.318485 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:32.318516 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:09:32.318966 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:32.318968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:32.319010 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:32.319186 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:32.320161 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.321443 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.322559 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.323043 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.323585 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:32.327561 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:32.327568 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:32.327568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:32.327584 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:32.330762 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331340 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:32.331343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:32.331344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:32.331457 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331765 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:32.331766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:32.331766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:32.332046 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:32.332141 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:32.332448 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 00:09:32.332591 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":63,"threads_started":1}
I20260812 00:09:32.332715 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.318183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34141 (local address 127.7.85.193:33449)
0812 00:09:32.318343 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:32.318354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:32.318398 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:32.319215 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:32.319218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:32.319686 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:32.320002 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:32.320010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.321548 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.321550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:32.322454 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:32.322457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.323172 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.323173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:32.323264 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:32.324017 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:32.324027 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:32.327406 (+ 3379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:32.330896 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:32.330909 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:32.330912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:32.331230 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:32.331537 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:32.331539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:32.331689 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:32.332133 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:32.332134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:32.332273 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 00:09:32.332386 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 00:09:32.332603 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":32,"thread_start_us":73,"threads_started":1}
I20260812 00:09:32.332747 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.318297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:53563 (local address 127.7.85.194:35467)
0812 00:09:32.318540 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:32.318548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:32.318567 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:09:32.318981 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:32.318983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:32.319019 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:32.319186 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:32.320186 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.321570 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.322581 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.322999 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.323131 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:32.328116 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:32.328121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:32.328122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:32.328134 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:32.330930 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331406 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:32.331409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:32.331410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:32.331516 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331772 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:32.331773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:32.331774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:32.332046 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:32.332141 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:32.332454 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 00:09:32.332597 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":52,"threads_started":1}
I20260812 00:09:32.332835 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.318170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35467 (local address 127.7.85.193:53563)
0812 00:09:32.318315 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:32.318328 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:32.318398 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:32.319215 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:32.319218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:32.319686 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:32.320001 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:32.320010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.321709 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.321710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:32.322453 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:32.322456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:32.323105 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:32.323106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:32.323246 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:32.323972 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:32.323981 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:32.326979 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:32.331065 (+ 4086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:32.331067 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:32.331310 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:32.331592 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:32.331593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:32.331594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:32.331697 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:32.332124 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:32.332132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:32.332272 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 00:09:32.332385 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 00:09:32.332690 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":176,"thread_start_us":62,"threads_started":1}
W20260812 00:09:32.333763 3046 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
W20260812 00:09:32.333868 3045 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
I20260812 00:09:32.333927 3045 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de9e8fbebb704a8293f078551f000192; no voters: 8d3a413bfa7f49ebaf90aff344964e83, 8e66ebc7ed4e4285821aeab116a13368
I20260812 00:09:32.334100 3744 raft_consensus.cc:2726] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:09:32.597632 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.597107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:54543)
0812 00:09:32.597487 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:32.597564 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
I20260812 00:09:32.672559 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.672293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:56521)
0812 00:09:32.672442 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:32.672498 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260812 00:09:32.814234 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:32.813908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:42691)
0812 00:09:32.814092 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:32.814164 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":76,"threads_started":1}
I20260812 00:09:33.599119 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:33.598871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:36291)
0812 00:09:33.599023 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:33.599068 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260812 00:09:33.673470 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:33.673104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:48693)
0812 00:09:33.673306 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:33.673356 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":124,"threads_started":1}
I20260812 00:09:33.814971 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:33.814714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:49591)
0812 00:09:33.814869 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:33.814920 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260812 00:09:34.028415 3764 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:34.028519 3764 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:34.028752 3764 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141), 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
W20260812 00:09:34.029299 3046 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
W20260812 00:09:34.029425 3045 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
I20260812 00:09:34.029512 3045 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de9e8fbebb704a8293f078551f000192; no voters: 8d3a413bfa7f49ebaf90aff344964e83, 8e66ebc7ed4e4285821aeab116a13368
I20260812 00:09:34.029603 3764 raft_consensus.cc:2726] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:09:34.600019 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:34.599752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:53161)
0812 00:09:34.599917 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:34.599966 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260812 00:09:34.674271 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:34.674028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:53047)
0812 00:09:34.674163 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:34.674205 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260812 00:09:34.815922 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:34.815608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:37651)
0812 00:09:34.815825 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:34.815872 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
I20260812 00:09:35.575270 3771 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:35.575404 3771 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:35.576490 3771 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141), 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
W20260812 00:09:35.576629 3045 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
W20260812 00:09:35.576725 3046 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
I20260812 00:09:35.576797 3046 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de9e8fbebb704a8293f078551f000192; no voters: 8d3a413bfa7f49ebaf90aff344964e83, 8e66ebc7ed4e4285821aeab116a13368
I20260812 00:09:35.576915 3771 raft_consensus.cc:2726] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:09:35.601007 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:35.600743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:48087)
0812 00:09:35.600912 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:35.600953 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260812 00:09:35.675096 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:35.674818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:34309)
0812 00:09:35.674998 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:35.675044 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260812 00:09:35.816924 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:35.816582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:43979)
0812 00:09:35.816814 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:35.816864 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":105,"threads_started":1}
I20260812 00:09:36.601969 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:36.601646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:59619)
0812 00:09:36.601841 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:36.601889 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
I20260812 00:09:36.676813 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:36.676048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:35355)
0812 00:09:36.676610 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:36.676707 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":100,"threads_started":1}
I20260812 00:09:36.817857 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:36.817578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:38467)
0812 00:09:36.817752 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:36.817794 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":85,"threads_started":1}
I20260812 00:09:37.602949 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:37.602664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:39221)
0812 00:09:37.602829 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:37.602894 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260812 00:09:37.677784 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:37.677495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:44771)
0812 00:09:37.677681 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:37.677727 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":82,"threads_started":1}
I20260812 00:09:37.818850 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:37.818481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:55617)
0812 00:09:37.818674 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:37.818714 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260812 00:09:38.603943 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:38.603663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:37073)
0812 00:09:38.603840 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:38.603891 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":72,"threads_started":1}
I20260812 00:09:38.679378 3787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:38.678988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:59773)
0812 00:09:38.679217 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:38.679292 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":115,"threads_started":1}
I20260812 00:09:38.819936 3789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:38.819552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:39939)
0812 00:09:38.819788 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:38.819843 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":80,"threads_started":1}
I20260812 00:09:39.604921 3790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:39.604667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:33227)
0812 00:09:39.604823 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:39.604864 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260812 00:09:39.680325 3792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:39.680036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:53125)
0812 00:09:39.680236 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:39.680273 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":102,"threads_started":1}
I20260812 00:09:39.820607 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:39.820384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:33449)
0812 00:09:39.820509 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:39.820554 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260812 00:09:40.455116 3797 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:40.455236 3797 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:40.455457 3797 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141), 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
W20260812 00:09:40.456099 3046 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
W20260812 00:09:40.456269 3045 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
I20260812 00:09:40.456333 3045 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de9e8fbebb704a8293f078551f000192; no voters: 8d3a413bfa7f49ebaf90aff344964e83, 8e66ebc7ed4e4285821aeab116a13368
I20260812 00:09:40.456426 3797 raft_consensus.cc:2726] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:09:40.605892 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:40.605605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:57999)
0812 00:09:40.605784 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:40.605837 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
I20260812 00:09:40.681356 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:40.681022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:39105)
0812 00:09:40.681258 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:40.681306 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":95,"threads_started":1}
I20260812 00:09:40.821530 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:40.821260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:57677)
0812 00:09:40.821420 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:40.821472 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260812 00:09:41.606997 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:41.606707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:58133)
0812 00:09:41.606894 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:41.606942 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260812 00:09:41.682117 3803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:41.681900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:41679)
0812 00:09:41.682038 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:41.682080 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260812 00:09:41.822592 3804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:41.822304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:34743)
0812 00:09:41.822487 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:41.822534 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260812 00:09:42.056735 3806 raft_consensus.cc:482] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:42.056857 3806 raft_consensus.cc:504] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d3a413bfa7f49ebaf90aff344964e83" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34141 } } peers { permanent_uuid: "8e66ebc7ed4e4285821aeab116a13368" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35467 } } peers { permanent_uuid: "de9e8fbebb704a8293f078551f000192" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 41713 } }
I20260812 00:09:42.057054 3806 leader_election.cc:288] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141), 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467)
W20260812 00:09:42.057659 3046 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8d3a413bfa7f49ebaf90aff344964e83 (127.7.85.195:34141): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
W20260812 00:09:42.057857 3045 leader_election.cc:341] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8e66ebc7ed4e4285821aeab116a13368 (127.7.85.194:35467): Not found: Tablet not found: 3794414e0582468994ab7998c4302a3b
I20260812 00:09:42.057958 3045 leader_election.cc:302] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de9e8fbebb704a8293f078551f000192; no voters: 8d3a413bfa7f49ebaf90aff344964e83, 8e66ebc7ed4e4285821aeab116a13368
I20260812 00:09:42.058115 3806 raft_consensus.cc:2726] T 3794414e0582468994ab7998c4302a3b P de9e8fbebb704a8293f078551f000192 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:09:42.607822 3807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:42.607541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:53877)
0812 00:09:42.607720 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:42.607763 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":66,"threads_started":1}
I20260812 00:09:42.682840 3809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:42.682599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:47747)
0812 00:09:42.682754 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:42.682800 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260812 00:09:42.823540 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:42.823268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:44081)
0812 00:09:42.823436 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:42.823482 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260812 00:09:43.608839 3815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:43.608553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:41303)
0812 00:09:43.608744 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:43.608784 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":122,"threads_started":1}
I20260812 00:09:43.683641 3816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:43.683409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:36631)
0812 00:09:43.683563 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:43.683603 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":64,"threads_started":1}
I20260812 00:09:43.824469 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:43.824153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:40485)
0812 00:09:43.824334 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:43.824406 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260812 00:09:44.609721 3818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:44.609452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:39433)
0812 00:09:44.609631 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:44.609670 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260812 00:09:44.684547 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:44.684285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:52149)
0812 00:09:44.684447 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:44.684487 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260812 00:09:44.825433 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:44.825113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:58303)
0812 00:09:44.825324 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:44.825369 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":142,"threads_started":1}
I20260812 00:09:45.610862 3824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:45.610570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.193:38311)
0812 00:09:45.610752 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:45.610798 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260812 00:09:45.685470 3827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:45.685130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.194:39071)
0812 00:09:45.685360 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:45.685409 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":150,"threads_started":1}
I20260812 00:09:45.826372 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:45.826095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33245 (local address 127.7.85.195:43743)
0812 00:09:45.826254 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:45.826302 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:33245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260812 00:09:45.829803 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 2968
I20260812 00:09:45.845472 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 3260
I20260812 00:09:45.863559 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 3504
2026-08-12T00:09:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19127 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:09:45 dist-test-slave-z8x9 krb5kdc[3833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:09:45 dist-test-slave-z8x9 krb5kdc[3833](info): set up 2 sockets
Aug 12 00:09:45 dist-test-slave-z8x9 krb5kdc[3833](info): commencing operation
krb5kdc: starting...
W20260812 00:09:47.231280 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:09:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:09:47.377427 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37003
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:37003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:47.468379 3855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:47.468647 3855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:47.468675 3855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:47.470094 3855 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 00:09:47.470158 3855 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 00:09:47.470171 3855 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 00:09:47.470184 3855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:09:47.470197 3855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:47.470211 3855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:47.470224 3855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:09:47.470283 3855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:09:47.471992 3855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:37003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:47.472325 3855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:47.472599 3855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:47.479519 3863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:47.479734 3861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:47.479768 3871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:47.480118 3855 server_base.cc:1034] running on GCE node
I20260812 00:09:47.480460 3855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:47.480765 3855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:47.481930 3855 hybrid_clock.cc:648] HybridClock initialized: now 1786493387481919 us; error 36 us; skew 500 ppm
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:47.485019 3855 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:09:47.485611 3855 webserver.cc:460] Webserver started at http://127.7.85.254:35249/ using document root <none> and password file <none>
I20260812 00:09:47.485867 3855 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:47.485939 3855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:47.486096 3855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:47.487205 3855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "93706a51a9bd4108a44b1bfc7aa258f6"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "f358732d5bb966adb27074e9b20fb243"
server_key_iv: "ac587cbe7ab375a7e2076296e2c9687d"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.487587 3855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "93706a51a9bd4108a44b1bfc7aa258f6"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "f358732d5bb966adb27074e9b20fb243"
server_key_iv: "ac587cbe7ab375a7e2076296e2c9687d"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.488997 3855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:09:47.489989 3881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.490195 3855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:47.490321 3855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "93706a51a9bd4108a44b1bfc7aa258f6"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "f358732d5bb966adb27074e9b20fb243"
server_key_iv: "ac587cbe7ab375a7e2076296e2c9687d"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.490422 3855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:47.515098 3855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:47.517661 3855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:47.517863 3855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:47.522822 3855 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:37003
I20260812 00:09:47.522869 3991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:37003 every 8 connection(s)
I20260812 00:09:47.523305 3855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:09:47.524356 3992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:47.524724 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 3855
I20260812 00:09:47.524864 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:09:47.525107 7511 external_mini_cluster.cc:1428] Setting key d972590771934c87985a5ec398259869
I20260812 00:09:47.527947 3992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap starting.
I20260812 00:09:47.528656 3992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:47.529102 3992 log.cc:824] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:47.529994 3992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: No bootstrap required, opened a new log
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:47.532682 3992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:09:47.532874 3992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:47.532950 3992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93706a51a9bd4108a44b1bfc7aa258f6, State: Initialized, Role: FOLLOWER
I20260812 00:09:47.533095 3992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:09:47.533219 3992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:09:47.533270 3992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:09:47.533322 3992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:47.534106 3992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:09:47.534245 3992 leader_election.cc:302] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93706a51a9bd4108a44b1bfc7aa258f6; no voters:
I20260812 00:09:47.534471 3992 leader_election.cc:288] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:09:47.534675 3999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:47.534773 3992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:09:47.535090 3999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 LEADER]: Becoming Leader. State: Replica: 93706a51a9bd4108a44b1bfc7aa258f6, State: Running, Role: LEADER
I20260812 00:09:47.535386 3999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:09:47.535907 4001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93706a51a9bd4108a44b1bfc7aa258f6. Latest consensus state: current_term: 1 leader_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } } }
I20260812 00:09:47.536007 4001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:47.536062 4000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } } }
I20260812 00:09:47.536140 4000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:09:47.536283 4013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:09:47.536890 4013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:09:47.538331 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:47.525806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.7.85.254:37003)
0812 00:09:47.526003 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:47.526010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:47.526037 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:09:47.526597 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:47.526606 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:47.526661 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:47.526806 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:47.527669 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.528678 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.529663 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.530292 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.530417 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:47.533426 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:47.533431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:47.533431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:47.533446 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:47.535671 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:47.536707 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:47.536711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:47.536711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:47.536840 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:47.537076 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:47.537077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:47.537078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:47.537462 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:47.537622 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:47.537929 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 00:09:47.538080 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260812 00:09:47.539299 4013 catalog_manager.cc:1347] Generated new cluster ID: 9df29be1f7e54856a1dab73315af3f18
I20260812 00:09:47.539399 4013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:09:47.554210 4013 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:09:47.554819 4013 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:09:47.557845 4013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Generated new TSK 0
I20260812 00:09:47.558099 4013 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:09:47.593358 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37003
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:47.683388 4042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:47.683672 4042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:47.683717 4042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:47.684955 4042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:47.685038 4042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:47.685088 4042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:09:47.687250 4042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:47.687589 4042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:47.687858 4042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:47.693636 4051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:47.693658 4052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:47.693768 4054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:47.694202 4042 server_base.cc:1034] running on GCE node
I20260812 00:09:47.694428 4042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:47.694653 4042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:47.695847 4042 hybrid_clock.cc:648] HybridClock initialized: now 1786493387695767 us; error 94 us; skew 500 ppm
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:47.699015 4042 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:09:47.699550 4042 webserver.cc:460] Webserver started at http://127.7.85.193:42295/ using document root <none> and password file <none>
I20260812 00:09:47.699801 4042 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:47.699851 4042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:47.699985 4042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:47.701085 4042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "f2ecd2c1710f430ebc1f440a87d5022b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "52a6a122b542fa07721ea5ee3577cb44"
server_key_iv: "11929f35823eb0eb2df8bdb3f5ec69d8"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.701560 4042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "f2ecd2c1710f430ebc1f440a87d5022b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "52a6a122b542fa07721ea5ee3577cb44"
server_key_iv: "11929f35823eb0eb2df8bdb3f5ec69d8"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.703040 4042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:47.703995 4066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.704218 4042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:47.704329 4042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "f2ecd2c1710f430ebc1f440a87d5022b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "52a6a122b542fa07721ea5ee3577cb44"
server_key_iv: "11929f35823eb0eb2df8bdb3f5ec69d8"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.704402 4042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:47.724128 4042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:47.726853 4042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:47.726989 4042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:47.727217 4042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:47.727581 4042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:47.727618 4042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.727667 4042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:47.727695 4042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.736522 4042 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:35889
I20260812 00:09:47.736601 4289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:35889 every 8 connection(s)
I20260812 00:09:47.737736 4042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:09:47.740841 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 4042
I20260812 00:09:47.741020 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:09:47.741300 7511 external_mini_cluster.cc:1428] Setting key 788c8b089f68d02d58348fc41f5de16e
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:47.749785 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:47.738825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:40501 (local address 127.7.85.254:37003)
0812 00:09:47.738932 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:47.738935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:47.739310 (+ 375us) server_negotiation.cc:407] Connection header received
0812 00:09:47.740079 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:47.740081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:47.740095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:47.740179 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:47.741462 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.742003 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.743079 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.743294 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.743380 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:47.746046 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:47.746048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:47.746049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:47.746058 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:47.747985 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:47.748553 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:47.748559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:47.748559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:47.748638 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:47.748932 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:47.748933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:47.748934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:47.749073 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:47.749161 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:47.749540 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 00:09:47.749684 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:09:47.749992 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:47.738763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:40501)
0812 00:09:47.739055 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:47.739116 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:47.739897 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:47.740269 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:47.740273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:47.740961 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:47.741337 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:47.741343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.742117 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.742118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:47.742974 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:47.742977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.743394 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.743395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:47.743545 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:47.744164 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:47.744173 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:47.745925 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:47.748097 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:47.748108 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:47.748110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:47.748425 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:47.748724 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:47.748725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:47.748726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:47.748856 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:47.749156 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:47.749210 (+ 54us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:47.749383 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 00:09:47.749505 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 00:09:47.749721 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":131,"threads_started":1}
I20260812 00:09:47.750782 4293 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37003
I20260812 00:09:47.750941 4293 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:47.751155 4293 heartbeater.cc:511] Master 127.7.85.254:37003 requested a full tablet report, sending...
I20260812 00:09:47.751855 3913 ts_manager.cc:194] Registered new tserver with Master: f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889)
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:09:47.752872 3913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:40501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:09:47.777868 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37003
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:47.879102 4304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:47.879379 4304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:47.879410 4304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:47.880684 4304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:47.880735 4304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:47.880765 4304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:09:47.882524 4304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:47.882768 4304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:47.882979 4304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:47.888751 4314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:47.888799 4315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:47.888799 4318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:47.889063 4304 server_base.cc:1034] running on GCE node
I20260812 00:09:47.889364 4304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:47.889663 4304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:47.890858 4304 hybrid_clock.cc:648] HybridClock initialized: now 1786493387890842 us; error 32 us; skew 500 ppm
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:47.899189 4304 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:09:47.899837 4304 webserver.cc:460] Webserver started at http://127.7.85.194:46815/ using document root <none> and password file <none>
I20260812 00:09:47.900173 4304 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:47.900228 4304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:47.900375 4304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:47.901612 4304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "fcbcb98b58174c5b8fe1473e5c018c5b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "b6b571e39fbd7d3727634626410ba95e"
server_key_iv: "6eb60bd315e2295f97365388b0b36f97"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.902015 4304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "fcbcb98b58174c5b8fe1473e5c018c5b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "b6b571e39fbd7d3727634626410ba95e"
server_key_iv: "6eb60bd315e2295f97365388b0b36f97"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.903615 4304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:09:47.904744 4327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.905000 4304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:47.905100 4304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "fcbcb98b58174c5b8fe1473e5c018c5b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "b6b571e39fbd7d3727634626410ba95e"
server_key_iv: "6eb60bd315e2295f97365388b0b36f97"
server_key_version: "encryptionkey@0"
I20260812 00:09:47.905216 4304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:47.929014 4304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:47.932650 4304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:47.932802 4304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:47.933027 4304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:47.933455 4304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:47.933497 4304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.933522 4304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:47.933537 4304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:47.940725 4304 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35303
I20260812 00:09:47.940773 4542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35303 every 8 connection(s)
I20260812 00:09:47.941141 4304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:09:47.947466 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 4304
I20260812 00:09:47.947621 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:09:47.947841 7511 external_mini_cluster.cc:1428] Setting key 9c9f5bc9b597571d0d496c0c6b218374
Aug 12 00:09:47 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:47.951984 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:47.942110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:44021 (local address 127.7.85.254:37003)
0812 00:09:47.942235 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:47.942238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:47.942454 (+ 216us) server_negotiation.cc:407] Connection header received
0812 00:09:47.943030 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:47.943032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:47.943046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:47.943099 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:47.944149 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.944667 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.945789 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.946001 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.946095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:47.948771 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:47.948773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:47.948774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:47.948782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:47.950324 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:47.950814 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:47.950815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:47.950816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:47.950885 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:47.951163 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:47.951164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:47.951164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:47.951300 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:47.951383 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:47.951776 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 00:09:47.951879 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 00:09:47.952204 4545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:47.942035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:44021)
0812 00:09:47.942319 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:47.942362 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:47.942912 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:47.943178 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:47.943181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:47.943819 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:47.944031 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:47.944035 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.944779 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.944781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:47.945687 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:47.945690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:47.946115 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:47.946116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:47.946256 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:47.946877 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:47.946886 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:47.948675 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:47.950460 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:47.950469 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:47.950472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:47.950729 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:47.950965 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:47.950965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:47.950966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:47.951087 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:47.951373 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:47.951381 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:47.951598 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 00:09:47.951751 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 00:09:47.951950 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":100,"threads_started":1}
I20260812 00:09:47.952806 4543 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37003
I20260812 00:09:47.952925 4543 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:47.953135 4543 heartbeater.cc:511] Master 127.7.85.254:37003 requested a full tablet report, sending...
I20260812 00:09:47.953877 3913 ts_manager.cc:194] Registered new tserver with Master: fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303)
I20260812 00:09:47.954483 3913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:44021
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:09:47.981936 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37003
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:09:48.072083 4549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:09:48.072332 4549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:09:48.072359 4549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:09:48.073789 4549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:09:48.073853 4549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:09:48.073884 4549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:09:48.075500 4549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:09:48.075734 4549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:09:48.075933 4549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:09:48.081961 4566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:48.082007 4569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:09:48.082056 4567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:09:48.082219 4549 server_base.cc:1034] running on GCE node
I20260812 00:09:48.082440 4549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:09:48.082708 4549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:09:48.083890 4549 hybrid_clock.cc:648] HybridClock initialized: now 1786493388083878 us; error 46 us; skew 500 ppm
Aug 12 00:09:48 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493388, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:09:48.086730 4549 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:09:48.087237 4549 webserver.cc:460] Webserver started at http://127.7.85.195:46517/ using document root <none> and password file <none>
I20260812 00:09:48.087459 4549 fs_manager.cc:359] Metadata directory not provided
I20260812 00:09:48.087502 4549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:09:48.087594 4549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:09:48.089022 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "33252da37d00420a85f3e06462708300"
format_stamp: "Formatted at 2026-08-12 00:09:48 on dist-test-slave-z8x9"
server_key: "f9b985dd9637a399217c7fb02f2dc6bc"
server_key_iv: "7c6f389717009e95e7601af60f796ff2"
server_key_version: "encryptionkey@0"
I20260812 00:09:48.089469 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "33252da37d00420a85f3e06462708300"
format_stamp: "Formatted at 2026-08-12 00:09:48 on dist-test-slave-z8x9"
server_key: "f9b985dd9637a399217c7fb02f2dc6bc"
server_key_iv: "7c6f389717009e95e7601af60f796ff2"
server_key_version: "encryptionkey@0"
I20260812 00:09:48.090879 4549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:48.091763 4582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:48.092059 4549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:09:48.092197 4549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "33252da37d00420a85f3e06462708300"
format_stamp: "Formatted at 2026-08-12 00:09:48 on dist-test-slave-z8x9"
server_key: "f9b985dd9637a399217c7fb02f2dc6bc"
server_key_iv: "7c6f389717009e95e7601af60f796ff2"
server_key_version: "encryptionkey@0"
I20260812 00:09:48.092295 4549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:09:48.108953 4549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:09:48.112150 4549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:09:48.112401 4549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:09:48.112684 4549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:09:48.113075 4549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:09:48.113134 4549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:48.113171 4549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:09:48.113253 4549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:09:48.120229 4549 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34069
I20260812 00:09:48.120272 4812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34069 every 8 connection(s)
I20260812 00:09:48.120621 4549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
Aug 12 00:09:48 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493388, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:09:48.128839 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 4549
I20260812 00:09:48.128938 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:09:48.129137 7511 external_mini_cluster.cc:1428] Setting key d393aff7bc1d89b30b56559a0507ec96
I20260812 00:09:48.131680 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.121456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:53655 (local address 127.7.85.254:37003)
0812 00:09:48.121538 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.121540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.121771 (+ 231us) server_negotiation.cc:407] Connection header received
0812 00:09:48.122270 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.122271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.122282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.122339 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:48.123352 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.124043 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.125107 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.125401 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.125470 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.127930 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:48.127932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:48.127932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:48.127940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:48.130150 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:48.130771 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:48.130772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:48.130772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:48.130843 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:48.131089 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:48.131090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:48.131090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:48.131208 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:48.131261 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.131525 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.131608 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 00:09:48.131857 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.121420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:53655)
0812 00:09:48.121648 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.121695 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:48.122178 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:48.122430 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:48.122433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:48.123041 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:09:48.123249 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.123255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.124174 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.124175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.125005 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.125008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.125497 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.125498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.125614 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.126271 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:09:48.126280 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:09:48.127827 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:09:48.130324 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:48.130336 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:48.130339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:48.130655 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:48.130915 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:09:48.130916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:09:48.130916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:09:48.131028 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:09:48.131252 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:09:48.131255 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:09:48.131376 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 00:09:48.131456 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:09:48.131596 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":114,"threads_started":1}
I20260812 00:09:48.132515 4815 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37003
I20260812 00:09:48.132643 4815 heartbeater.cc:464] Registering TS with master...
I20260812 00:09:48.132862 4815 heartbeater.cc:511] Master 127.7.85.254:37003 requested a full tablet report, sending...
I20260812 00:09:48.133345 3913 ts_manager.cc:194] Registered new tserver with Master: 33252da37d00420a85f3e06462708300 (127.7.85.195:34069)
I20260812 00:09:48.133692 3913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:53655
I20260812 00:09:48.141831 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:09:48.150033 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.143392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.7.85.254:37003)
0812 00:09:48.143480 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.143483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.143590 (+ 107us) server_negotiation.cc:407] Connection header received
0812 00:09:48.143682 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.143683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.143696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.143754 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:09:48.144369 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.144877 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.145705 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.145916 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.145995 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.147331 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:09:48.147333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:09:48.147333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:09:48.147341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:09:48.148700 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:48.149095 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:48.149097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:48.149097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:48.149164 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:09:48.149423 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:09:48.149425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:09:48.149425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:09:48.149601 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:09:48.149718 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.149829 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.149919 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 00:09:48.151829 3913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:09:48.153573 3913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:09:48.161383 4835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.156322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34069 (local address 127.0.0.1:45250)
0812 00:09:48.156568 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.156615 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:48.156694 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:48.157014 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:48.157018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:48.157020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:48.157105 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.157107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.158364 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.158365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.160065 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.160068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.161063 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.161064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.161129 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.161143 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 00:09:48.161259 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 00:09:48.161312 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"spinlock_wait_cycles":896,"thread_start_us":144,"threads_started":1}
I20260812 00:09:48.161415 4840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.156546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.7.85.195:34069)
0812 00:09:48.156833 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.156837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.156845 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:09:48.156878 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.156880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.156917 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.156997 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:48.157412 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.158224 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.160169 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.160961 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.161044 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.161104 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.161277 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.161324 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":100,"threads_started":1}
I20260812 00:09:48.161834 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.156341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35303 (local address 127.0.0.1:51324)
0812 00:09:48.156843 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.156853 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:48.156909 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:48.157224 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:48.157225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:48.157227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:48.157266 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.157270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.159283 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.159284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.160573 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.160576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.161589 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.161590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.161656 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.161662 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:09:48.161710 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 00:09:48.161753 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":78,"thread_start_us":111,"threads_started":1}
I20260812 00:09:48.162089 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.156614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.7.85.194:35303)
0812 00:09:48.156803 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.156807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.156967 (+ 160us) server_negotiation.cc:407] Connection header received
0812 00:09:48.157015 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.157017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.157033 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.157107 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:48.157337 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.158452 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.160693 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.161500 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.161576 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.161633 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.161953 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.162005 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260812 00:09:48.162490 4680 tablet_service.cc:1467] Processing CreateTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 (DEFAULT_TABLE table=test-table [id=3007062317b94561ba466160ccf4b751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:48.162720 4413 tablet_service.cc:1467] Processing CreateTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 (DEFAULT_TABLE table=test-table [id=3007062317b94561ba466160ccf4b751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:48.162945 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.156323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:35889 (local address 127.0.0.1:34094)
0812 00:09:48.156825 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.156837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:48.156909 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:48.157203 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:48.157205 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:48.157207 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:48.157265 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.157268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.159283 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.159284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.160573 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.160576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.162687 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.162688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.162756 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.162760 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:09:48.162829 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 00:09:48.162872 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":123,"thread_start_us":49,"threads_started":1}
I20260812 00:09:48.162953 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.156553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.7.85.193:35889)
0812 00:09:48.156755 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.156761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.156930 (+ 169us) server_negotiation.cc:407] Connection header received
0812 00:09:48.156986 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.156989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.157003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.157078 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:48.158371 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.159145 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.161674 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.162587 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.162686 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.162739 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.162836 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.162876 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":81,"threads_started":1}
I20260812 00:09:48.163021 4680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b48338a480d403a9c71679dae8ec2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:48.163147 4413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b48338a480d403a9c71679dae8ec2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:48.163533 4154 tablet_service.cc:1467] Processing CreateTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 (DEFAULT_TABLE table=test-table [id=3007062317b94561ba466160ccf4b751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:09:48.163769 4154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b48338a480d403a9c71679dae8ec2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:09:48.166675 4844 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap starting.
I20260812 00:09:48.167243 4848 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Bootstrap starting.
I20260812 00:09:48.167510 4844 tablet_bootstrap.cc:654] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:48.167863 4848 tablet_bootstrap.cc:654] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:48.168084 4844 log.cc:824] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:48.168504 4848 log.cc:824] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:48.168879 4851 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Bootstrap starting.
I20260812 00:09:48.168949 4844 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: No bootstrap required, opened a new log
I20260812 00:09:48.169039 4844 ts_tablet_manager.cc:1397] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:09:48.169310 4848 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: No bootstrap required, opened a new log
I20260812 00:09:48.169386 4848 ts_tablet_manager.cc:1397] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:09:48.169600 4851 tablet_bootstrap.cc:654] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Neither blocks nor log segments found. Creating new log.
I20260812 00:09:48.170221 4851 log.cc:824] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:09:48.171118 4851 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: No bootstrap required, opened a new log
I20260812 00:09:48.171042 4844 raft_consensus.cc:348] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.171205 4851 ts_tablet_manager.cc:1397] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:09:48.171209 4844 raft_consensus.cc:374] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:48.171248 4844 raft_consensus.cc:729] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ecd2c1710f430ebc1f440a87d5022b, State: Initialized, Role: FOLLOWER
I20260812 00:09:48.171397 4844 consensus_queue.cc:260] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.171777 4844 ts_tablet_manager.cc:1428] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:09:48.171713 4848 raft_consensus.cc:348] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.171871 4848 raft_consensus.cc:374] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:48.171916 4848 raft_consensus.cc:729] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33252da37d00420a85f3e06462708300, State: Initialized, Role: FOLLOWER
I20260812 00:09:48.171954 4293 heartbeater.cc:503] Master 127.7.85.254:37003 was elected leader, sending a full tablet report...
I20260812 00:09:48.172187 4848 consensus_queue.cc:260] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.172696 4848 ts_tablet_manager.cc:1428] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 00:09:48.172726 4815 heartbeater.cc:503] Master 127.7.85.254:37003 was elected leader, sending a full tablet report...
I20260812 00:09:48.173574 4851 raft_consensus.cc:348] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.173789 4851 raft_consensus.cc:374] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:09:48.173842 4851 raft_consensus.cc:729] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbcb98b58174c5b8fe1473e5c018c5b, State: Initialized, Role: FOLLOWER
I20260812 00:09:48.173960 4851 consensus_queue.cc:260] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.174299 4851 ts_tablet_manager.cc:1428] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:09:48.174374 4543 heartbeater.cc:503] Master 127.7.85.254:37003 was elected leader, sending a full tablet report...
W20260812 00:09:48.191854 4544 tablet.cc:2369] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:09:48.238700 4294 tablet.cc:2369] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:09:48.309441 4861 raft_consensus.cc:482] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:09:48.309592 4861 raft_consensus.cc:504] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.310026 4861 leader_election.cc:288] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33252da37d00420a85f3e06462708300 (127.7.85.195:34069), fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303)
I20260812 00:09:48.313268 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.310241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34069 (local address 127.7.85.193:55727)
0812 00:09:48.310417 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.310431 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:48.310513 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:48.310679 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:48.310680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:48.310683 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:48.310775 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.310777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.311448 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.311449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.312356 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.312359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.312955 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.312956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.313035 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.313040 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:09:48.313095 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 00:09:48.313127 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260812 00:09:48.313362 4840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.310351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:55727 (local address 127.7.85.195:34069)
0812 00:09:48.310401 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.310403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.310517 (+ 114us) server_negotiation.cc:407] Connection header received
0812 00:09:48.310582 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.310583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.310597 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.310680 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:48.310860 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.311359 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.312464 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.312863 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.312921 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.312928 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.313239 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.313292 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:09:48.313428 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.310242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35303 (local address 127.7.85.193:41129)
0812 00:09:48.310435 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.310445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:09:48.310536 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:09:48.310717 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:09:48.310718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:09:48.310721 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:09:48.310848 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.310851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.311509 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.311510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.312367 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:09:48.312370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.313031 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.313032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:09:48.313229 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.313235 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:09:48.313293 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 00:09:48.313339 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260812 00:09:48.313436 4843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.310378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:41129 (local address 127.7.85.194:35303)
0812 00:09:48.310453 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.310456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.310521 (+ 65us) server_negotiation.cc:407] Connection header received
0812 00:09:48.310613 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.310614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.310627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.310688 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:09:48.310932 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.311411 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.312517 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.312925 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.313003 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.313012 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.313304 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.313358 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:09:48.313745 4717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b48338a480d403a9c71679dae8ec2b3" candidate_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33252da37d00420a85f3e06462708300" is_pre_election: true
I20260812 00:09:48.313874 4717 raft_consensus.cc:2454] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ecd2c1710f430ebc1f440a87d5022b in term 0.
I20260812 00:09:48.313884 4448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b48338a480d403a9c71679dae8ec2b3" candidate_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" is_pre_election: true
I20260812 00:09:48.314039 4448 raft_consensus.cc:2454] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ecd2c1710f430ebc1f440a87d5022b in term 0.
I20260812 00:09:48.314226 4070 leader_election.cc:302] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33252da37d00420a85f3e06462708300, f2ecd2c1710f430ebc1f440a87d5022b; no voters:
I20260812 00:09:48.314491 4861 raft_consensus.cc:2781] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:09:48.314558 4861 raft_consensus.cc:482] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:09:48.314576 4861 raft_consensus.cc:3037] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:48.315315 4861 raft_consensus.cc:504] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.315472 4861 leader_election.cc:288] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 1 election: Requested vote from peers 33252da37d00420a85f3e06462708300 (127.7.85.195:34069), fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303)
I20260812 00:09:48.315667 4448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b48338a480d403a9c71679dae8ec2b3" candidate_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b"
I20260812 00:09:48.315773 4448 raft_consensus.cc:3037] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:48.315845 4717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b48338a480d403a9c71679dae8ec2b3" candidate_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33252da37d00420a85f3e06462708300"
I20260812 00:09:48.315944 4717 raft_consensus.cc:3037] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:09:48.316670 4717 raft_consensus.cc:2454] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ecd2c1710f430ebc1f440a87d5022b in term 1.
I20260812 00:09:48.316658 4448 raft_consensus.cc:2454] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ecd2c1710f430ebc1f440a87d5022b in term 1.
I20260812 00:09:48.316918 4070 leader_election.cc:302] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33252da37d00420a85f3e06462708300, f2ecd2c1710f430ebc1f440a87d5022b; no voters:
I20260812 00:09:48.317068 4861 raft_consensus.cc:2781] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:09:48.317299 4861 raft_consensus.cc:686] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 LEADER]: Becoming Leader. State: Replica: f2ecd2c1710f430ebc1f440a87d5022b, State: Running, Role: LEADER
I20260812 00:09:48.317380 4861 consensus_queue.cc:237] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:09:48.318300 3913 catalog_manager.cc:5495] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b reported cstate change: term changed from 0 to 1, leader changed from <none> to f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193). New cstate: current_term: 1 leader_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } health_report { overall_health: UNKNOWN } } }
I20260812 00:09:48.346313 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.343732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.7.85.193:35889)
0812 00:09:48.343816 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 00:09:48.343818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:09:48.343828 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:09:48.343883 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:09:48.343884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:09:48.343907 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:09:48.343989 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:09:48.344133 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.344611 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.345484 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:09:48.345688 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:09:48.345770 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:09:48.345950 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:09:48.346066 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:09:48.346166 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 00:09:48.346186 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 00:09:48.346209 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:09:48.348573 4717 raft_consensus.cc:1261] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Refusing update from remote peer f2ecd2c1710f430ebc1f440a87d5022b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:48.348680 4448 raft_consensus.cc:1261] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Refusing update from remote peer f2ecd2c1710f430ebc1f440a87d5022b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:09:48.348991 4868 consensus_queue.cc:1035] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [LEADER]: Connected to new peer: Peer: permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:48.349135 4861 consensus_queue.cc:1035] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:09:48.354362 4876 mvcc.cc:205] Tried to move back new op lower bound from 7317476918672445440 to 7317476918548520960. Current Snapshot: MvccSnapshot[applied={T|T < 7317476918672445440 or (T in {7317476918672445440})}]
I20260812 00:09:48.356309 3913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:48.356451 3913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:09:48.357403 3913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Sending DeleteTablet for 3 replicas of tablet 2b48338a480d403a9c71679dae8ec2b3
I20260812 00:09:48.357833 4413 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:48 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:51324
I20260812 00:09:48.357903 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 4042
I20260812 00:09:48.357894 4154 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:48 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:34094
I20260812 00:09:48.358461 4680 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:48 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:45250
I20260812 00:09:48.365530 4890 tablet_replica.cc:330] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: stopping tablet replica
I20260812 00:09:48.365526 4891 tablet_replica.cc:330] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: stopping tablet replica
I20260812 00:09:48.365715 4890 raft_consensus.cc:2229] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:48.365756 4891 raft_consensus.cc:2229] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:09:48.365872 4890 raft_consensus.cc:2258] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:48.366020 4891 raft_consensus.cc:2258] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:09:48.368470 4891 ts_tablet_manager.cc:1905] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:48.370151 4890 ts_tablet_manager.cc:1905] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:09:48.370623 4891 ts_tablet_manager.cc:1918] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:48.370800 4891 log.cc:1196] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/2b48338a480d403a9c71679dae8ec2b3
I20260812 00:09:48.371229 4891 ts_tablet_manager.cc:1939] T 2b48338a480d403a9c71679dae8ec2b3 P 33252da37d00420a85f3e06462708300: Deleting consensus metadata
I20260812 00:09:48.371846 3887 catalog_manager.cc:4841] TS 33252da37d00420a85f3e06462708300 (127.7.85.195:34069): tablet 2b48338a480d403a9c71679dae8ec2b3 (table test-table [id=3007062317b94561ba466160ccf4b751]) successfully deleted
I20260812 00:09:48.372231 4890 ts_tablet_manager.cc:1918] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:09:48.372330 4890 log.cc:1196] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/2b48338a480d403a9c71679dae8ec2b3
I20260812 00:09:48.372632 4890 ts_tablet_manager.cc:1939] T 2b48338a480d403a9c71679dae8ec2b3 P fcbcb98b58174c5b8fe1473e5c018c5b: Deleting consensus metadata
I20260812 00:09:48.373100 3885 catalog_manager.cc:4841] TS fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303): tablet 2b48338a480d403a9c71679dae8ec2b3 (table test-table [id=3007062317b94561ba466160ccf4b751]) successfully deleted
W20260812 00:09:48.375802 3886 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:35889 (error 108)
I20260812 00:09:48.375941 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 4304
I20260812 00:09:48.383101 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:09:48.382898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:35889 (local address 127.0.0.1:34108)
0812 00:09:48.382976 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 00:09:48.383042 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:35889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 00:09:48.383311 3886 catalog_manager.cc:4584] TS f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b48338a480d403a9c71679dae8ec2b3: Network error: Client connection negotiation failed: client connection to 127.7.85.193:35889: connect: Connection refused (error 111)
I20260812 00:09:48.393949 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 4549
I20260812 00:09:48.404500 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 3855
I20260812 00:10:08.420495 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37003
--webserver_interface=127.7.85.254
--webserver_port=35249
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:37003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:08.512347 4894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:08.512583 4894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:08.512609 4894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:08.513872 4894 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 00:10:08.513923 4894 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 00:10:08.513937 4894 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 00:10:08.513949 4894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:08.513962 4894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:08.513976 4894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:08.513988 4894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:08.514001 4894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:08.515599 4894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:37003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=35249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:08.515890 4894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:08.516144 4894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:08.527051 4909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.527052 4912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.527098 4917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:08.527091 4894 server_base.cc:1034] running on GCE node
I20260812 00:10:08.527647 4894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:08.528013 4894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:08.529215 4894 hybrid_clock.cc:648] HybridClock initialized: now 1786493408529155 us; error 30 us; skew 500 ppm
Aug 12 00:10:08 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:08.536826 4894 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:10:08.537734 4894 webserver.cc:460] Webserver started at http://127.7.85.254:35249/ using document root <none> and password file <none>
I20260812 00:10:08.538040 4894 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:08.538220 4894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:08.540460 4894 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:10:08.541384 4928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:08.541610 4894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:08.541738 4894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "93706a51a9bd4108a44b1bfc7aa258f6"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "f358732d5bb966adb27074e9b20fb243"
server_key_iv: "ac587cbe7ab375a7e2076296e2c9687d"
server_key_version: "encryptionkey@0"
I20260812 00:10:08.542230 4894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:08.549250 4894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:08.551709 4894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:08.551913 4894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:08.556836 4894 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:37003
I20260812 00:10:08.556967 5023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:37003 every 8 connection(s)
I20260812 00:10:08.557389 4894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:08.558409 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 4894
I20260812 00:10:08.558763 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:35889
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:37003
--webserver_port=42295
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:10:08.559653 5025 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:10:08.560818 5025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap starting.
I20260812 00:10:08.570837 5025 log.cc:824] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:08.573137 5025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:10:08.573597 5025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap complete.
I20260812 00:10:08.576128 5025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:08.576382 5025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93706a51a9bd4108a44b1bfc7aa258f6, State: Initialized, Role: FOLLOWER
I20260812 00:10:08.576558 5025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:08.576629 5025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:08.576689 5025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:08.576742 5025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:10:08.577906 5025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:08.577997 5025 leader_election.cc:302] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93706a51a9bd4108a44b1bfc7aa258f6; no voters:
I20260812 00:10:08.578253 5025 leader_election.cc:288] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:10:08.578290 5033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:10:08.578410 5033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 LEADER]: Becoming Leader. State: Replica: 93706a51a9bd4108a44b1bfc7aa258f6, State: Running, Role: LEADER
I20260812 00:10:08.578557 5025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:08.578563 5033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:08.579020 5036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } } }
I20260812 00:10:08.579102 5036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:08.579520 5048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:08.579571 5038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93706a51a9bd4108a44b1bfc7aa258f6. Latest consensus state: current_term: 2 leader_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } } }
I20260812 00:10:08.579703 5038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:08.580724 5048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:08.581030 5048 catalog_manager.cc:1259] Loaded cluster ID: 9df29be1f7e54856a1dab73315af3f18
I20260812 00:10:08.581077 5048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:08.581920 5048 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:08.582134 5048 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Loaded TSK: 0
I20260812 00:10:08.590481 5048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Generated new TSK 1
I20260812 00:10:08.590615 5048 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:10:08.661186 5028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:08.661537 5028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:08.661583 5028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:08.663880 5028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:08.663966 5028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:08.664021 5028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:08.666177 5028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:35889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=42295
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:08.666498 5028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:08.666803 5028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:08.673710 5079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.673776 5080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:08.673882 5028 server_base.cc:1034] running on GCE node
W20260812 00:10:08.674296 5082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:08.674536 5028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:08.674927 5028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:08.676122 5028 hybrid_clock.cc:648] HybridClock initialized: now 1786493408676106 us; error 28 us; skew 500 ppm
Aug 12 00:10:08 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:08.680078 5028 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:10:08.680764 5028 webserver.cc:460] Webserver started at http://127.7.85.193:42295/ using document root <none> and password file <none>
I20260812 00:10:08.681100 5028 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:08.681227 5028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:08.683158 5028 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:10:08.684056 5093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:08.684394 5028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:08.684502 5028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "f2ecd2c1710f430ebc1f440a87d5022b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "52a6a122b542fa07721ea5ee3577cb44"
server_key_iv: "11929f35823eb0eb2df8bdb3f5ec69d8"
server_key_version: "encryptionkey@0"
I20260812 00:10:08.684990 5028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:08.707728 5028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:08.710729 5028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:08.710919 5028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:08.711215 5028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:08.711892 5105 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 00:10:08.713752 5028 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 00:10:08.713820 5028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 00:10:08.713867 5028 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 00:10:08.714745 5028 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 00:10:08.714797 5028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 00:10:08.714911 5105 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap starting.
I20260812 00:10:08.722512 5028 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:35889
I20260812 00:10:08.722820 5306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:35889 every 8 connection(s)
I20260812 00:10:08.723012 5028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:08.726053 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 5028
I20260812 00:10:08.726405 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:35303
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:37003
--webserver_port=46815
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:10:08.728664 5105 log.cc:824] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:08.732390 5105 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:10:08.732817 5105 tablet_bootstrap.cc:492] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap complete.
Aug 12 00:10:08 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:08.734390 5105 ts_tablet_manager.cc:1397] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.006s
I20260812 00:10:08.736899 5105 raft_consensus.cc:348] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:10:08.737375 5105 raft_consensus.cc:729] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ecd2c1710f430ebc1f440a87d5022b, State: Initialized, Role: FOLLOWER
I20260812 00:10:08.737569 5105 consensus_queue.cc:260] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } }
I20260812 00:10:08.737916 5105 ts_tablet_manager.cc:1428] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260812 00:10:08.738838 5312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:08.723895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:48353)
0812 00:10:08.724107 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:08.724153 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:08.724992 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:08.725832 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:08.725838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:08.726412 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:08.727040 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:08.727049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.728948 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.728950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:08.730222 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:08.730227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.731163 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.731164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:08.731304 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:08.732102 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:08.732123 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:08.734196 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:08.736625 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:08.736633 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:08.736635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:08.736942 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:08.737287 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:08.737288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:08.737289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:08.737560 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:08.738143 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:08.738150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:08.738299 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 00:10:08.738413 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 00:10:08.738597 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":121,"threads_started":1}
I20260812 00:10:08.738977 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:08.723996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:48353 (local address 127.7.85.254:37003)
0812 00:10:08.725320 (+ 1324us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:08.725329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:08.725373 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:10:08.725511 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:08.725518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:08.725573 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:08.725719 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:08.727217 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.728797 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.730341 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.730931 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.731020 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:08.734346 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:08.734352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:08.734352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:08.734367 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:08.736522 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:08.737042 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:08.737045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:08.737046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:08.737202 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:08.737643 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:08.737644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:08.737644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:08.737954 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:08.738013 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:08.738527 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 00:10:08.738702 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1268,"thread_start_us":93,"threads_started":1}
I20260812 00:10:08.740104 5307 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37003
I20260812 00:10:08.740222 5307 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:08.740447 5307 heartbeater.cc:511] Master 127.7.85.254:37003 requested a full tablet report, sending...
I20260812 00:10:08.741537 4959 ts_manager.cc:194] Registered new tserver with Master: f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889)
I20260812 00:10:08.742946 4959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:48353
I20260812 00:10:08.744002 5307 heartbeater.cc:503] Master 127.7.85.254:37003 was elected leader, sending a full tablet report...
I20260812 00:10:08.749749 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:08.742318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:35889 (local address 127.0.0.1:38432)
0812 00:10:08.745305 (+ 2987us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:08.745347 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:08.745419 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:08.745781 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:08.745785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:08.745787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:08.745868 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:08.745871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.747120 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.747121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:08.748386 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:08.748389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.749438 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.749439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:08.749524 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:08.749538 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 00:10:08.749615 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 00:10:08.749667 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2950,"thread_start_us":98,"threads_started":1}
I20260812 00:10:08.749765 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:08.742594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.7.85.193:35889)
0812 00:10:08.745552 (+ 2958us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:08.745558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:08.745566 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:10:08.745668 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:08.745670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:08.745685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:08.745758 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:08.745953 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.747041 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.748489 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.749345 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.749423 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:08.749480 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:08.749643 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 00:10:08.749687 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2925,"thread_start_us":76,"threads_started":1}
I20260812 00:10:08.750231 5182 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b48338a480d403a9c71679dae8ec2b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:09:48 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:38432
I20260812 00:10:08.750814 5323 tablet_replica.cc:330] stopping tablet replica
I20260812 00:10:08.750931 5323 raft_consensus.cc:2229] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:08.751003 5323 raft_consensus.cc:2258] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:08.751345 5323 ts_tablet_manager.cc:1905] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:08.754169 5323 ts_tablet_manager.cc:1918] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:08.754262 5323 log.cc:1196] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal/wals/2b48338a480d403a9c71679dae8ec2b3
I20260812 00:10:08.754495 5323 ts_tablet_manager.cc:1939] T 2b48338a480d403a9c71679dae8ec2b3 P f2ecd2c1710f430ebc1f440a87d5022b: Deleting consensus metadata
I20260812 00:10:08.755072 4932 catalog_manager.cc:4841] TS f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889): tablet 2b48338a480d403a9c71679dae8ec2b3 (table test-table [id=3007062317b94561ba466160ccf4b751]) successfully deleted
W20260812 00:10:08.824942 5317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:08.825246 5317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:08.825302 5317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:08.826579 5317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:08.826659 5317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:08.826702 5317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:08.828297 5317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:35303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=46815
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:08.828609 5317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:08.828871 5317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:08.835095 5340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.835176 5338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.835189 5343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:08.835466 5317 server_base.cc:1034] running on GCE node
I20260812 00:10:08.835675 5317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:08.836068 5317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:08.837375 5317 hybrid_clock.cc:648] HybridClock initialized: now 1786493408837329 us; error 65 us; skew 500 ppm
Aug 12 00:10:08 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:08.840274 5317 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:10:08.840862 5317 webserver.cc:460] Webserver started at http://127.7.85.194:46815/ using document root <none> and password file <none>
I20260812 00:10:08.841102 5317 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:08.841233 5317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:08.842779 5317 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:08.843559 5362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:08.843775 5317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:08.843883 5317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "fcbcb98b58174c5b8fe1473e5c018c5b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "b6b571e39fbd7d3727634626410ba95e"
server_key_iv: "6eb60bd315e2295f97365388b0b36f97"
server_key_version: "encryptionkey@0"
I20260812 00:10:08.844276 5317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:08.852991 5317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:08.855544 5317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:08.855746 5317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:08.856022 5317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:08.856710 5317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:08.856787 5317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:08.856837 5317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:08.856858 5317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:08.863682 5317 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35303
I20260812 00:10:08.863729 5577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35303 every 8 connection(s)
I20260812 00:10:08.864114 5317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:08.865208 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 5317
I20260812 00:10:08.865604 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:34069
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:37003
--webserver_port=46517
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:10:08 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:08.877043 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:08.864971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:45285 (local address 127.7.85.254:37003)
0812 00:10:08.865047 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:08.865049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:08.865349 (+ 300us) server_negotiation.cc:407] Connection header received
0812 00:10:08.866172 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:08.866174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:08.866190 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:08.866250 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:08.867735 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.868580 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.869832 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.870176 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.870261 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:08.873038 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:08.873040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:08.873040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:08.873050 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:08.874975 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:08.875474 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:08.875476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:08.875477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:08.875551 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:08.875822 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:08.875823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:08.875824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:08.876001 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:08.876086 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:08.876772 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 00:10:08.876893 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:10:08.877116 5581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:08.864947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:45285)
0812 00:10:08.865187 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:08.865241 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:08.866033 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:08.866359 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:08.866364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:08.867270 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:08.867588 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:08.867596 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.868720 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.868721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:08.869733 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:08.869737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:08.870372 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:08.870375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:08.870676 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:08.871408 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:08.871417 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:08.872921 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:08.875090 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:08.875098 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:08.875101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:08.875389 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:08.875631 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:08.875632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:08.875632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:08.875748 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:08.876168 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:08.876185 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:08.876442 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 00:10:08.876735 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 00:10:08.876933 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260812 00:10:08.877801 5578 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37003
I20260812 00:10:08.877924 5578 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:08.878151 5578 heartbeater.cc:511] Master 127.7.85.254:37003 requested a full tablet report, sending...
I20260812 00:10:08.878602 4959 ts_manager.cc:194] Registered new tserver with Master: fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303)
I20260812 00:10:08.879207 4959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:45285
W20260812 00:10:08.957346 5582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:08.957574 5582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:08.957600 5582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:08.958798 5582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:08.958846 5582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:08.958874 5582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:08.960397 5582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:34069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=46517
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:08.960620 5582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:08.960811 5582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:10:08.966668 5582 server_base.cc:1034] running on GCE node
W20260812 00:10:08.966614 5593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.966645 5596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:08.966614 5592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:08.967051 5582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:08.967342 5582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:08.968545 5582 hybrid_clock.cc:648] HybridClock initialized: now 1786493408968509 us; error 54 us; skew 500 ppm
Aug 12 00:10:08 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:08.971781 5582 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:10:08.972333 5582 webserver.cc:460] Webserver started at http://127.7.85.195:46517/ using document root <none> and password file <none>
I20260812 00:10:08.972615 5582 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:08.972721 5582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:08.974270 5582 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:08.975062 5607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:08.975340 5582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:10:08.975455 5582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "33252da37d00420a85f3e06462708300"
format_stamp: "Formatted at 2026-08-12 00:09:48 on dist-test-slave-z8x9"
server_key: "f9b985dd9637a399217c7fb02f2dc6bc"
server_key_iv: "7c6f389717009e95e7601af60f796ff2"
server_key_version: "encryptionkey@0"
I20260812 00:10:08.976114 5582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:08.997110 5582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:08.999639 5582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:08.999859 5582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:09.000138 5582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:09.000484 5582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:09.000542 5582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:09.000586 5582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:09.000630 5582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:09.007916 5582 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34069
I20260812 00:10:09.007978 5833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34069 every 8 connection(s)
I20260812 00:10:09.008391 5582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:09.013588 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 5582
Aug 12 00:10:09 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:09.019675 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.009260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:56357)
0812 00:10:09.009410 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:09.009448 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:09.009928 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:09.010165 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:09.010168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:09.010786 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:09.010987 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.010991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.011709 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.011710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.012540 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.012542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.012956 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.012957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.013058 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.013788 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:09.013797 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:09.015963 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:09.017969 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:09.017978 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:09.017981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:09.018224 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:09.018452 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:09.018453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:09.018453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:09.018606 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:09.019019 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:09.019026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:09.019156 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 00:10:09.019299 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 00:10:09.019491 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260812 00:10:09.019667 5313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.009229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:56357 (local address 127.7.85.254:37003)
0812 00:10:09.009290 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.009292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.009566 (+ 274us) server_negotiation.cc:407] Connection header received
0812 00:10:09.010041 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.010043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.010054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.010114 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:09.011088 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.011615 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.012664 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.012869 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.012937 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.016074 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:09.016080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:09.016081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:09.016087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:09.017844 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:09.018313 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:09.018314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:09.018315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:09.018381 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:09.018705 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:09.018706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:09.018707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:09.018930 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:09.019022 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.019372 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.019547 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:10:09.020231 5834 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37003
I20260812 00:10:09.020362 5834 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:09.020601 5834 heartbeater.cc:511] Master 127.7.85.254:37003 requested a full tablet report, sending...
I20260812 00:10:09.021071 4959 ts_manager.cc:194] Registered new tserver with Master: 33252da37d00420a85f3e06462708300 (127.7.85.195:34069)
I20260812 00:10:09.021579 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.014148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.7.85.254:37003)
0812 00:10:09.014317 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.014319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.014407 (+ 88us) server_negotiation.cc:407] Connection header received
0812 00:10:09.014518 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.014519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.014535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.014619 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:09.015464 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.016038 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.016860 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.017049 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.017128 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.018356 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:09.018358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:09.018358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:09.018366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:09.020202 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:09.020603 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:09.020604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:09.020605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:09.020671 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:09.020945 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:09.020945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:09.020946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:09.021078 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:09.021285 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.021440 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.021513 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260812 00:10:09.021737 4959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:56357
I20260812 00:10:09.022507 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:09.025730 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.022938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.7.85.254:37003)
0812 00:10:09.023043 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.023045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.023152 (+ 107us) server_negotiation.cc:407] Connection header received
0812 00:10:09.023295 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.023296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.023334 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.023420 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:09.023553 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.024042 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.024837 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.025054 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.025134 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.025343 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:09.025400 (+ 57us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 00:10:09.025656 (+ 256us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59786: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 00:10:09.025807 5839 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59786: token expired
W20260812 00:10:09.025934 4823 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.85.254:37003)
I20260812 00:10:09.032336 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.026223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59796 (local address 127.7.85.254:37003)
0812 00:10:09.026283 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.026285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.026327 (+ 42us) server_negotiation.cc:407] Connection header received
0812 00:10:09.026348 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.026348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.026358 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.026407 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:09.026902 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.027361 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.028067 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.028272 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.028355 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.029677 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:09.029678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:09.029679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:09.029686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:09.031096 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:09.031472 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:09.031473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:09.031473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:09.031539 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:09.031758 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:09.031759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:09.031759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:09.031908 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:09.032026 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.032125 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.032212 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 00:10:09.033757 4959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:09.035503 4959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:09.038058 5182 tablet_service.cc:1467] Processing CreateTablet for tablet ae854022ee324ad684aeecc16a1bfa42 (DEFAULT_TABLE table=test-table [id=e89a8aa39cc14af7acd1cdcc09cf3bd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:09.038300 5182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae854022ee324ad684aeecc16a1bfa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:09.039981 5105 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap starting.
I20260812 00:10:09.040319 5105 tablet_bootstrap.cc:654] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:09.040874 5105 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: No bootstrap required, opened a new log
I20260812 00:10:09.040930 5105 ts_tablet_manager.cc:1397] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 00:10:09.041044 5105 raft_consensus.cc:348] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.041152 5105 raft_consensus.cc:374] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:09.041235 5105 raft_consensus.cc:729] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ecd2c1710f430ebc1f440a87d5022b, State: Initialized, Role: FOLLOWER
I20260812 00:10:09.041287 5105 consensus_queue.cc:260] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.041388 5105 ts_tablet_manager.cc:1428] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 00:10:09.042313 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.037896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.7.85.195:34069)
0812 00:10:09.038187 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.038193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.038203 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:10:09.038245 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.038247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.038265 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.038353 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:09.038605 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.039517 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.040867 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.041840 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.041940 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.042013 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.042180 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.042225 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":126,"threads_started":1}
I20260812 00:10:09.042305 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.037743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34069 (local address 127.0.0.1:44554)
0812 00:10:09.037983 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:09.037994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:09.038050 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:09.038396 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:09.038398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:09.038401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:09.038462 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.038464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.039609 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.039610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.040771 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.040774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.042019 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.042020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.042095 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.042100 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:10:09.042159 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 00:10:09.042225 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":58,"thread_start_us":112,"threads_started":1}
I20260812 00:10:09.042408 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.037744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35303 (local address 127.0.0.1:44376)
0812 00:10:09.037785 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:09.037797 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:09.037836 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:09.038404 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:09.038406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:09.038408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:09.038457 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.038459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.039784 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.039785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.040828 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.040830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.042106 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.042107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.042173 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.042203 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 00:10:09.042256 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 00:10:09.042305 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":23}
I20260812 00:10:09.042418 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.038020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.7.85.194:35303)
0812 00:10:09.038206 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.038211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.038223 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:10:09.038265 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.038267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.038283 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.038536 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:09.038623 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.039694 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.040964 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.041961 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.042104 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.042174 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.042288 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.042331 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260812 00:10:09.043121 5708 tablet_service.cc:1467] Processing CreateTablet for tablet ae854022ee324ad684aeecc16a1bfa42 (DEFAULT_TABLE table=test-table [id=e89a8aa39cc14af7acd1cdcc09cf3bd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:09.043138 5443 tablet_service.cc:1467] Processing CreateTablet for tablet ae854022ee324ad684aeecc16a1bfa42 (DEFAULT_TABLE table=test-table [id=e89a8aa39cc14af7acd1cdcc09cf3bd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:09.043524 5708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae854022ee324ad684aeecc16a1bfa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:09.043525 5443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae854022ee324ad684aeecc16a1bfa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:09.047534 5864 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Bootstrap starting.
I20260812 00:10:09.047608 5863 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Bootstrap starting.
I20260812 00:10:09.048357 5864 tablet_bootstrap.cc:654] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:09.048358 5863 tablet_bootstrap.cc:654] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:09.049083 5863 log.cc:824] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:09.049083 5864 log.cc:824] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:09.050083 5864 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: No bootstrap required, opened a new log
I20260812 00:10:09.050175 5864 ts_tablet_manager.cc:1397] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:10:09.050316 5863 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: No bootstrap required, opened a new log
I20260812 00:10:09.050391 5863 ts_tablet_manager.cc:1397] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:10:09.051806 5864 raft_consensus.cc:348] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.051832 5863 raft_consensus.cc:348] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.051970 5864 raft_consensus.cc:374] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:09.051996 5864 raft_consensus.cc:729] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33252da37d00420a85f3e06462708300, State: Initialized, Role: FOLLOWER
I20260812 00:10:09.051995 5863 raft_consensus.cc:374] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:09.052037 5863 raft_consensus.cc:729] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcbcb98b58174c5b8fe1473e5c018c5b, State: Initialized, Role: FOLLOWER
I20260812 00:10:09.052134 5864 consensus_queue.cc:260] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.052156 5863 consensus_queue.cc:260] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.052488 5863 ts_tablet_manager.cc:1428] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:10:09.052486 5864 ts_tablet_manager.cc:1428] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:10:09.052525 5834 heartbeater.cc:503] Master 127.7.85.254:37003 was elected leader, sending a full tablet report...
I20260812 00:10:09.052582 5578 heartbeater.cc:503] Master 127.7.85.254:37003 was elected leader, sending a full tablet report...
W20260812 00:10:09.114799 5579 tablet.cc:2369] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:10:09.224184 5308 tablet.cc:2369] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:10:09.259063 5836 tablet.cc:2369] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:10:09.259276 5870 raft_consensus.cc:482] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:09.259433 5870 raft_consensus.cc:504] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.259894 5870 leader_election.cc:288] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33252da37d00420a85f3e06462708300 (127.7.85.195:34069), f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889)
I20260812 00:10:09.263069 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.260237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:37803 (local address 127.7.85.195:34069)
0812 00:10:09.260299 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.260301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.260459 (+ 158us) server_negotiation.cc:407] Connection header received
0812 00:10:09.260552 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.260553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.260565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.260627 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:09.260818 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.261353 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.262281 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.262688 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.262775 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.262784 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.262940 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.262981 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 00:10:09.263074 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.260282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:59101 (local address 127.7.85.193:35889)
0812 00:10:09.260413 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.260415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.260422 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:10:09.260449 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.260449 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.260459 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.260529 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:09.260706 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.261260 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.262241 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.262659 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.262750 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.262761 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.262936 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.262986 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260812 00:10:09.263091 5581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.260135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:35889 (local address 127.7.85.194:59101)
0812 00:10:09.260203 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:09.260219 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:09.260289 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:09.260549 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:09.260550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:09.260552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:09.260629 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.260631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.261345 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.261346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.262121 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.262123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.262744 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.262746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.262838 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.262841 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:09.262925 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:10:09.262978 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":5}
I20260812 00:10:09.263185 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.260146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34069 (local address 127.7.85.194:37803)
0812 00:10:09.260367 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:09.260383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:09.260474 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:09.260630 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:09.260631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:09.260633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:09.260730 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.260732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.261438 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.261439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.262181 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:09.262183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.262775 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.262776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:09.262861 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.262864 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:09.262934 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 00:10:09.262981 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":17,"thread_start_us":142,"threads_started":1}
I20260812 00:10:09.263567 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae854022ee324ad684aeecc16a1bfa42" candidate_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33252da37d00420a85f3e06462708300" is_pre_election: true
I20260812 00:10:09.263756 5744 raft_consensus.cc:2454] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcbcb98b58174c5b8fe1473e5c018c5b in term 0.
I20260812 00:10:09.263816 5231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae854022ee324ad684aeecc16a1bfa42" candidate_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" is_pre_election: true
I20260812 00:10:09.264015 5231 raft_consensus.cc:2454] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcbcb98b58174c5b8fe1473e5c018c5b in term 0.
I20260812 00:10:09.264015 5368 leader_election.cc:302] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33252da37d00420a85f3e06462708300, fcbcb98b58174c5b8fe1473e5c018c5b; no voters:
I20260812 00:10:09.264243 5870 raft_consensus.cc:2781] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:09.264305 5870 raft_consensus.cc:482] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:09.264322 5870 raft_consensus.cc:3037] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:09.265097 5870 raft_consensus.cc:504] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.265445 5231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae854022ee324ad684aeecc16a1bfa42" candidate_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2ecd2c1710f430ebc1f440a87d5022b"
I20260812 00:10:09.265460 5744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae854022ee324ad684aeecc16a1bfa42" candidate_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33252da37d00420a85f3e06462708300"
I20260812 00:10:09.265561 5744 raft_consensus.cc:3037] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:09.265564 5231 raft_consensus.cc:3037] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:09.266254 5744 raft_consensus.cc:2454] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcbcb98b58174c5b8fe1473e5c018c5b in term 1.
I20260812 00:10:09.266445 5368 leader_election.cc:302] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33252da37d00420a85f3e06462708300, fcbcb98b58174c5b8fe1473e5c018c5b; no voters:
I20260812 00:10:09.266474 5870 leader_election.cc:288] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [CANDIDATE]: Term 1 election: Requested vote from peers 33252da37d00420a85f3e06462708300 (127.7.85.195:34069), f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889)
I20260812 00:10:09.266531 5870 raft_consensus.cc:2781] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:09.266527 5231 raft_consensus.cc:2454] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcbcb98b58174c5b8fe1473e5c018c5b in term 1.
I20260812 00:10:09.266571 5870 raft_consensus.cc:686] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 LEADER]: Becoming Leader. State: Replica: fcbcb98b58174c5b8fe1473e5c018c5b, State: Running, Role: LEADER
I20260812 00:10:09.266670 5870 consensus_queue.cc:237] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:09.267612 4954 catalog_manager.cc:5495] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b reported cstate change: term changed from 0 to 1, leader changed from <none> to fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194). New cstate: current_term: 1 leader_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } health_report { overall_health: UNKNOWN } } }
I20260812 00:10:09.328970 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.326097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.7.85.194:35303)
0812 00:10:09.326204 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:09.326206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:09.326215 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:10:09.326256 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:09.326257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:09.326282 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:09.326362 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:09.326527 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.327008 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.327805 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:09.328062 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:09.328169 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:09.328506 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:09.328642 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:09.328724 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:09.328806 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 00:10:09.328833 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:10:09.331153 5231 raft_consensus.cc:1261] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Refusing update from remote peer fcbcb98b58174c5b8fe1473e5c018c5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:09.331342 5744 raft_consensus.cc:1261] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Refusing update from remote peer fcbcb98b58174c5b8fe1473e5c018c5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:09.331673 5877 consensus_queue.cc:1035] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:09.331902 5870 consensus_queue.cc:1035] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:09.334762 5884 mvcc.cc:205] Tried to move back new op lower bound from 7317477004617068544 to 7317477004356820992. Current Snapshot: MvccSnapshot[applied={T|T < 7317477004617068544}]
I20260812 00:10:09.335671 5890 mvcc.cc:205] Tried to move back new op lower bound from 7317477004617068544 to 7317477004356820992. Current Snapshot: MvccSnapshot[applied={T|T < 7317477004617068544}]
I20260812 00:10:09.338922 4959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:09.339054 4959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:09.340029 4959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Sending DeleteTablet for 3 replicas of tablet ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:09.340406 5182 tablet_service.cc:1514] Processing DeleteTablet for tablet ae854022ee324ad684aeecc16a1bfa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:09 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:38432
I20260812 00:10:09.340421 5443 tablet_service.cc:1514] Processing DeleteTablet for tablet ae854022ee324ad684aeecc16a1bfa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:09 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:44376
I20260812 00:10:09.340451 5708 tablet_service.cc:1514] Processing DeleteTablet for tablet ae854022ee324ad684aeecc16a1bfa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:09 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:44554
I20260812 00:10:09.340704 5903 tablet_replica.cc:330] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: stopping tablet replica
I20260812 00:10:09.340850 5902 tablet_replica.cc:330] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: stopping tablet replica
I20260812 00:10:09.340862 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 5028
I20260812 00:10:09.355779 5902 raft_consensus.cc:2229] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:10:09.355989 5902 raft_consensus.cc:2258] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:09.358321 5902 ts_tablet_manager.cc:1905] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:09.360821 5902 ts_tablet_manager.cc:1918] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:09.360932 5902 log.cc:1196] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:09.361280 5902 ts_tablet_manager.cc:1939] T ae854022ee324ad684aeecc16a1bfa42 P fcbcb98b58174c5b8fe1473e5c018c5b: Deleting consensus metadata
I20260812 00:10:09.361805 4929 catalog_manager.cc:4841] TS fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303): tablet ae854022ee324ad684aeecc16a1bfa42 (table test-table [id=e89a8aa39cc14af7acd1cdcc09cf3bd6]) successfully deleted
I20260812 00:10:09.340874 5903 raft_consensus.cc:2229] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:09.368129 5903 raft_consensus.cc:2258] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:09.368724 5903 ts_tablet_manager.cc:1905] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:09.370797 5903 ts_tablet_manager.cc:1918] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:09.370898 5903 log.cc:1196] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:09.371208 5903 ts_tablet_manager.cc:1939] T ae854022ee324ad684aeecc16a1bfa42 P 33252da37d00420a85f3e06462708300: Deleting consensus metadata
I20260812 00:10:09.371750 4933 catalog_manager.cc:4841] TS 33252da37d00420a85f3e06462708300 (127.7.85.195:34069): tablet ae854022ee324ad684aeecc16a1bfa42 (table test-table [id=e89a8aa39cc14af7acd1cdcc09cf3bd6]) successfully deleted
W20260812 00:10:09.373028 4932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:35889 (error 108)
I20260812 00:10:09.373541 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 5317
I20260812 00:10:09.373581 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:09.373430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:35889 (local address 127.0.0.1:38438)
0812 00:10:09.373492 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:09.373540 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:35889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 00:10:09.384701 4932 catalog_manager.cc:4584] TS f2ecd2c1710f430ebc1f440a87d5022b (127.7.85.193:35889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae854022ee324ad684aeecc16a1bfa42: Network error: Client connection negotiation failed: client connection to 127.7.85.193:35889: connect: Connection refused (error 111)
I20260812 00:10:09.392920 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 5582
I20260812 00:10:09.410714 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 4894
I20260812 00:10:29.428544 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37003
--webserver_interface=127.7.85.254
--webserver_port=35249
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:37003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:29.517747 5908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:29.518000 5908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:29.518024 5908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:29.519250 5908 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 00:10:29.519302 5908 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 00:10:29.519315 5908 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 00:10:29.519328 5908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:29.519341 5908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:29.519379 5908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:29.519410 5908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:29.519428 5908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:29.521031 5908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:37003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=35249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:29.521324 5908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:29.521584 5908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:29.527310 5917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:29.527177 5922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.527374 5908 server_base.cc:1034] running on GCE node
W20260812 00:10:29.527176 5918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.527956 5908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:29.528287 5908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:29.529592 5908 hybrid_clock.cc:648] HybridClock initialized: now 1786493429529571 us; error 41 us; skew 500 ppm
Aug 12 00:10:29 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:29.536473 5908 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:10:29.537015 5908 webserver.cc:460] Webserver started at http://127.7.85.254:35249/ using document root <none> and password file <none>
I20260812 00:10:29.537367 5908 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:29.537425 5908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:29.539166 5908 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:29.540349 5934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:29.540779 5908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:29.541013 5908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "93706a51a9bd4108a44b1bfc7aa258f6"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "f358732d5bb966adb27074e9b20fb243"
server_key_iv: "ac587cbe7ab375a7e2076296e2c9687d"
server_key_version: "encryptionkey@0"
I20260812 00:10:29.541540 5908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:29.560209 5908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:29.563627 5908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:29.564029 5908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:29.569716 5908 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:37003
I20260812 00:10:29.569756 6050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:37003 every 8 connection(s)
I20260812 00:10:29.570636 5908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:29.572541 6052 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:10:29.573339 6052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap starting.
I20260812 00:10:29.576380 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 5908
I20260812 00:10:29.576793 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:35889
--local_ip_for_outbound_sockets=127.7.85.193
--tserver_master_addrs=127.7.85.254:37003
--webserver_port=42295
--webserver_interface=127.7.85.193
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:10:29.584625 6052 log.cc:824] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:29.587019 6052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:10:29.587379 6052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Bootstrap complete.
I20260812 00:10:29.589883 6052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:29.590116 6052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93706a51a9bd4108a44b1bfc7aa258f6, State: Initialized, Role: FOLLOWER
I20260812 00:10:29.590301 6052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:29.590381 6052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:29.590440 6052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:29.590508 6052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:10:29.591566 6052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:29.591692 6052 leader_election.cc:302] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93706a51a9bd4108a44b1bfc7aa258f6; no voters:
I20260812 00:10:29.591909 6052 leader_election.cc:288] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 00:10:29.591949 6067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 00:10:29.592072 6067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [term 3 LEADER]: Becoming Leader. State: Replica: 93706a51a9bd4108a44b1bfc7aa258f6, State: Running, Role: LEADER
I20260812 00:10:29.592166 6067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } }
I20260812 00:10:29.592222 6052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:29.592444 6067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93706a51a9bd4108a44b1bfc7aa258f6. Latest consensus state: current_term: 3 leader_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } } }
I20260812 00:10:29.592526 6067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:29.592450 6068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93706a51a9bd4108a44b1bfc7aa258f6" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37003 } } }
I20260812 00:10:29.592715 6068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:29.592736 6076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:29.593743 6076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:29.593883 6076 catalog_manager.cc:1259] Loaded cluster ID: 9df29be1f7e54856a1dab73315af3f18
I20260812 00:10:29.594117 6076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:29.595033 6076 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:29.595263 6076 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Loaded TSK: 0
I20260812 00:10:29.595325 6076 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 93706a51a9bd4108a44b1bfc7aa258f6: Loaded TSK: 1
F20260812 00:10:29.595533 6076 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc6955219ad google::LogMessage::Fail() at ??:0
@ 0x7fc69552488a google::LogMessage::SendToLog() at ??:0
@ 0x7fc6955214de google::LogMessage::Flush() at ??:0
@ 0x7fc695522a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc697c8e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc697c8bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc69d47f1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc69d47f48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc69d47f594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc69d445c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc69d44fad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc6973ef0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc6973e4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc6982486db start_thread at ??:0
@ 0x7fc69401f61f clone at ??:0
W20260812 00:10:29.682483 6062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:29.682737 6062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:29.682777 6062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:29.683949 6062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:29.684017 6062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:29.684068 6062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:29.685680 6062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:35889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=42295
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:29.685954 6062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:29.686199 6062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:29.691543 6132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:29.691562 6137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.691713 6062 server_base.cc:1034] running on GCE node
W20260812 00:10:29.691636 6130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.692068 6062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:29.692355 6062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:29.693552 6062 hybrid_clock.cc:648] HybridClock initialized: now 1786493429693532 us; error 35 us; skew 500 ppm
Aug 12 00:10:29 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:29.696254 6062 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:10:29.696740 6062 webserver.cc:460] Webserver started at http://127.7.85.193:42295/ using document root <none> and password file <none>
I20260812 00:10:29.696996 6062 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:29.697045 6062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:29.698613 6062 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:29.699472 6147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:29.699875 6062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:29.700007 6062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "f2ecd2c1710f430ebc1f440a87d5022b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "52a6a122b542fa07721ea5ee3577cb44"
server_key_iv: "11929f35823eb0eb2df8bdb3f5ec69d8"
server_key_version: "encryptionkey@0"
I20260812 00:10:29.700400 6062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:29.714048 6062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:29.716590 6062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:29.716785 6062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:29.717036 6062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:29.717661 6163 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 00:10:29.719973 6062 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 00:10:29.720063 6062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 00:10:29.720117 6062 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 00:10:29.720803 6062 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 00:10:29.720860 6062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 00:10:29.720938 6163 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap starting.
I20260812 00:10:29.729305 6062 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:35889
I20260812 00:10:29.729380 6351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:35889 every 8 connection(s)
I20260812 00:10:29.729851 6062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:29.734051 6163 log.cc:824] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:29.734114 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 6062
I20260812 00:10:29.734453 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:35303
--local_ip_for_outbound_sockets=127.7.85.194
--tserver_master_addrs=127.7.85.254:37003
--webserver_port=46815
--webserver_interface=127.7.85.194
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:10:29.736305 6163 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:10:29.736660 6163 tablet_bootstrap.cc:492] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Bootstrap complete.
I20260812 00:10:29.738090 6163 ts_tablet_manager.cc:1397] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.014s
I20260812 00:10:29.740505 6163 raft_consensus.cc:348] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:29.740960 6163 raft_consensus.cc:729] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ecd2c1710f430ebc1f440a87d5022b, State: Initialized, Role: FOLLOWER
I20260812 00:10:29.741189 6163 consensus_queue.cc:260] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:29.741547 6163 ts_tablet_manager.cc:1428] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b: Time spent starting tablet: real 0.003s user 0.004s sys 0.001s
W20260812 00:10:29.817847 6358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:10:29.730654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:38521)
0812 00:10:29.730896 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:29.730959 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:29.731774 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:29.817670 (+ 85896us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":120,"threads_started":1}
W20260812 00:10:29.818140 6354 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:37003 (0 consecutive failures): Network error: Failed to ping master at 127.7.85.254:37003: Client connection negotiation failed: client connection to 127.7.85.254:37003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 00:10:29.818651 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:29.818432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:45179)
0812 00:10:29.818568 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:29.818624 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 00:10:29.819032 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:29.818955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:52313)
0812 00:10:29.818989 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:29.819014 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260812 00:10:29.819160 6354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:10:29.828969 6360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:29.829319 6360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:29.829385 6360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:29.830690 6360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:29.830775 6360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:29.830829 6360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:29.832515 6360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:35303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=46815
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:29.832820 6360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:29.833089 6360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:29.838814 6372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:29.838826 6374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:29.838999 6377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.839488 6360 server_base.cc:1034] running on GCE node
I20260812 00:10:29.839761 6360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:29.840179 6360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:29.841359 6360 hybrid_clock.cc:648] HybridClock initialized: now 1786493429841312 us; error 69 us; skew 500 ppm
Aug 12 00:10:29 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:29.844735 6360 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:10:29.845435 6360 webserver.cc:460] Webserver started at http://127.7.85.194:46815/ using document root <none> and password file <none>
I20260812 00:10:29.845721 6360 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:29.845775 6360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:29.847319 6360 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:10:29.848021 6398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:29.848219 6360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:29.848335 6360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "fcbcb98b58174c5b8fe1473e5c018c5b"
format_stamp: "Formatted at 2026-08-12 00:09:47 on dist-test-slave-z8x9"
server_key: "b6b571e39fbd7d3727634626410ba95e"
server_key_iv: "6eb60bd315e2295f97365388b0b36f97"
server_key_version: "encryptionkey@0"
I20260812 00:10:29.848716 6360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:29.865999 6360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:29.868558 6360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:29.868752 6360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:29.869037 6360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:29.869408 6360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:29.869464 6360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:29.869511 6360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:29.869556 6360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:29.877319 6360 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35303
I20260812 00:10:29.877393 6622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35303 every 8 connection(s)
I20260812 00:10:29.877766 6360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:29.879029 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:29.878599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:53123)
0812 00:10:29.878785 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:29.878865 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
W20260812 00:10:29.879582 6623 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:37003 (0 consecutive failures): Network error: Failed to ping master at 127.7.85.254:37003: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
I20260812 00:10:29.879914 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:29.879817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:44643)
0812 00:10:29.879864 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:29.879894 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 00:10:29.880417 6628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:29.880210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:47899)
0812 00:10:29.880362 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:29.880393 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260812 00:10:29.880602 6623 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:10:29.883066 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 6360
I20260812 00:10:29.883438 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:34069
--local_ip_for_outbound_sockets=127.7.85.195
--tserver_master_addrs=127.7.85.254:37003
--webserver_port=46517
--webserver_interface=127.7.85.195
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:29.973716 6629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:29.973960 6629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:29.973986 6629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:29.975646 6629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:29.975731 6629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:29.975767 6629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:29.978040 6629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:34069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=46517
--tserver_master_addrs=127.7.85.254:37003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:29.978354 6629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:29.978617 6629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:29.984699 6642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:29.984699 6643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.984802 6629 server_base.cc:1034] running on GCE node
W20260812 00:10:29.984962 6645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:29.985236 6629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:29.985532 6629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:29.986783 6629 hybrid_clock.cc:648] HybridClock initialized: now 1786493429986774 us; error 39 us; skew 500 ppm
Aug 12 00:10:29 dist-test-slave-z8x9 krb5kdc[3833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:29.990015 6629 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:10:29.990553 6629 webserver.cc:460] Webserver started at http://127.7.85.195:46517/ using document root <none> and password file <none>
I20260812 00:10:29.990824 6629 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:29.990902 6629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:29.992594 6629 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:29.993389 6667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:29.993682 6629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:29.993793 6629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "33252da37d00420a85f3e06462708300"
format_stamp: "Formatted at 2026-08-12 00:09:48 on dist-test-slave-z8x9"
server_key: "f9b985dd9637a399217c7fb02f2dc6bc"
server_key_iv: "7c6f389717009e95e7601af60f796ff2"
server_key_version: "encryptionkey@0"
I20260812 00:10:29.994171 6629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:30.024564 6629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:30.027137 6629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:30.027336 6629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:30.027619 6629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:30.027956 6629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:30.028008 6629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:30.028052 6629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:30.028091 6629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:30.034899 6629 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:34069
I20260812 00:10:30.034947 6900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:34069 every 8 connection(s)
I20260812 00:10:30.035387 6629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:30.036646 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.036201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:39911)
0812 00:10:30.036377 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.036463 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
W20260812 00:10:30.037123 6901 heartbeater.cc:644] Failed to heartbeat to 127.7.85.254:37003 (0 consecutive failures): Network error: Failed to ping master at 127.7.85.254:37003: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
I20260812 00:10:30.037519 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.037398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:47251)
0812 00:10:30.037460 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.037495 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 00:10:30.037864 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.037785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:46855)
0812 00:10:30.037824 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.037847 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 00:10:30.038021 6901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:10:30.040937 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 6629
I20260812 00:10:30.820077 6913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.819637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:39991)
0812 00:10:30.819974 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.820022 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":208,"threads_started":1}
I20260812 00:10:30.881475 6914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.881038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:38301)
0812 00:10:30.881364 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.881409 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":64,"threads_started":1}
I20260812 00:10:30.939234 6916 raft_consensus.cc:482] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:30.939420 6916 raft_consensus.cc:504] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:30.939783 6916 leader_election.cc:288] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303), 33252da37d00420a85f3e06462708300 (127.7.85.195:34069)
Aug 12 00:10:30 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.195@KRBTEST.COM
Aug 12 00:10:30 dist-test-slave-z8x9 krb5kdc[3833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.194@KRBTEST.COM
I20260812 00:10:30.951924 6919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.940104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:43601 (local address 127.7.85.195:34069)
0812 00:10:30.940287 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:30.940301 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:30.940324 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:10:30.940799 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:30.940801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:30.940856 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:30.941017 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:30.941909 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.942952 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.944082 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.944526 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.944653 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:30.947306 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:30.947310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:30.947310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:30.947321 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:30.949623 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:30.950238 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:30.950241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:30.950242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:30.950373 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:30.950727 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:30.950729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:30.950729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:30.951090 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:30.951184 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:30.951642 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 00:10:30.951805 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260812 00:10:30.951939 6918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.940103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:49025 (local address 127.7.85.194:35303)
0812 00:10:30.940303 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:30.940312 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:30.940342 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:10:30.940811 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:30.940814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:30.940856 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:30.941027 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:30.941908 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.942920 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.944108 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.944517 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.944639 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:30.947759 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:30.947762 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:30.947762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:30.947773 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:30.949904 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:30.950401 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:30.950404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:30.950404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:30.950518 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:30.950745 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:30.950746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:30.950746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:30.951081 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:30.951181 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:30.951636 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 00:10:30.951809 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":67,"threads_started":1}
I20260812 00:10:30.951924 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.940003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:34069 (local address 127.7.85.193:43601)
0812 00:10:30.940135 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.940145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:30.940189 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:30.941064 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:30.941067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:30.941573 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:30.941774 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:30.941782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.943107 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.943108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:30.943984 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:30.943987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.944618 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.944619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:30.944724 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:30.945426 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:30.945436 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:30.947175 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:30.949733 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:30.949741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:30.949743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:30.950138 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:30.950458 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:30.950458 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:30.950459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:30.950607 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:30.951189 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:30.951200 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:30.951375 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 00:10:30.951526 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 00:10:30.951776 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":63,"thread_start_us":66,"threads_started":1}
I20260812 00:10:30.951934 6913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:30.939974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35303 (local address 127.7.85.193:49025)
0812 00:10:30.940039 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:30.940051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:30.940112 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:30.941063 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:30.941066 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:30.941571 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:30.941774 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:30.941781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.943035 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.943036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:30.943988 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:30.943991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:30.944609 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:30.944611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:30.944723 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:30.945466 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:30.945471 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:30.947630 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:30.949990 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:30.949991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:30.949992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:30.950307 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:30.950592 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:30.950593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:30.950593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:30.950663 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:30.951189 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:30.951200 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:30.951375 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 00:10:30.951529 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 00:10:30.951836 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":198}
W20260812 00:10:30.952865 6148 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
W20260812 00:10:30.953583 6153 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33252da37d00420a85f3e06462708300 (127.7.85.195:34069): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:30.953635 6153 leader_election.cc:302] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2ecd2c1710f430ebc1f440a87d5022b; no voters: 33252da37d00420a85f3e06462708300, fcbcb98b58174c5b8fe1473e5c018c5b
I20260812 00:10:30.953799 6916 raft_consensus.cc:2726] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:10:31.038655 6920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:31.038413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:49503)
0812 00:10:31.038555 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:31.038609 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260812 00:10:31.820879 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:31.820594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:51673)
0812 00:10:31.820776 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:31.820823 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260812 00:10:31.882248 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:31.881983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:36189)
0812 00:10:31.882132 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:31.882201 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260812 00:10:32.039860 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:32.039494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:60323)
0812 00:10:32.039727 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:32.039792 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":104,"threads_started":1}
I20260812 00:10:32.821750 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:32.821480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:42601)
0812 00:10:32.821630 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:32.821698 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260812 00:10:32.883002 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:32.882735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:35309)
0812 00:10:32.882915 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:32.882954 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
I20260812 00:10:33.040773 6937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:33.040456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:33573)
0812 00:10:33.040682 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:33.040727 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":182,"threads_started":1}
I20260812 00:10:33.174067 6938 raft_consensus.cc:482] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:33.174175 6938 raft_consensus.cc:504] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:33.174376 6938 leader_election.cc:288] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303), 33252da37d00420a85f3e06462708300 (127.7.85.195:34069)
W20260812 00:10:33.174819 6148 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
W20260812 00:10:33.174902 6153 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33252da37d00420a85f3e06462708300 (127.7.85.195:34069): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:33.174995 6153 leader_election.cc:302] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2ecd2c1710f430ebc1f440a87d5022b; no voters: 33252da37d00420a85f3e06462708300, fcbcb98b58174c5b8fe1473e5c018c5b
I20260812 00:10:33.175083 6938 raft_consensus.cc:2726] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:10:33.822698 6941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:33.822410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:48507)
0812 00:10:33.822591 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:33.822642 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260812 00:10:33.883864 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:33.883591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:36451)
0812 00:10:33.883767 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:33.883813 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260812 00:10:34.041822 6944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:34.041374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:40793)
0812 00:10:34.041637 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:34.041749 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":68,"threads_started":1}
I20260812 00:10:34.823917 6946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:34.823514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:55691)
0812 00:10:34.823736 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:34.823861 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":68,"threads_started":1}
I20260812 00:10:34.880172 6947 raft_consensus.cc:482] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:34.880311 6947 raft_consensus.cc:504] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:34.880493 6947 leader_election.cc:288] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303), 33252da37d00420a85f3e06462708300 (127.7.85.195:34069)
W20260812 00:10:34.880901 6148 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
W20260812 00:10:34.881004 6153 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33252da37d00420a85f3e06462708300 (127.7.85.195:34069): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:34.881054 6153 leader_election.cc:302] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2ecd2c1710f430ebc1f440a87d5022b; no voters: 33252da37d00420a85f3e06462708300, fcbcb98b58174c5b8fe1473e5c018c5b
I20260812 00:10:34.881119 6947 raft_consensus.cc:2726] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:10:34.884814 6949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:34.884582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:38497)
0812 00:10:34.884725 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:34.884770 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260812 00:10:35.042668 6950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:35.042411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:34971)
0812 00:10:35.042544 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:35.042591 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260812 00:10:35.824782 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:35.824498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:35513)
0812 00:10:35.824670 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:35.824726 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260812 00:10:35.885756 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:35.885478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:39175)
0812 00:10:35.885655 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:35.885704 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260812 00:10:36.043606 6959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:36.043290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:44269)
0812 00:10:36.043446 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:36.043553 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260812 00:10:36.825738 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:36.825480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:51085)
0812 00:10:36.825637 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:36.825685 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260812 00:10:36.886606 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:36.886310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:48905)
0812 00:10:36.886500 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:36.886553 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":98,"threads_started":1}
I20260812 00:10:37.044517 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:37.044280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:40811)
0812 00:10:37.044428 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:37.044465 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260812 00:10:37.826653 6964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:37.826362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:41541)
0812 00:10:37.826542 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:37.826598 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260812 00:10:37.887507 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:37.887242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:49301)
0812 00:10:37.887407 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:37.887453 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
I20260812 00:10:38.045394 6968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:38.045095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:50449)
0812 00:10:38.045293 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:38.045332 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":113,"threads_started":1}
I20260812 00:10:38.660833 6969 raft_consensus.cc:482] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:38.660967 6969 raft_consensus.cc:504] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcbcb98b58174c5b8fe1473e5c018c5b" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35303 } } peers { permanent_uuid: "33252da37d00420a85f3e06462708300" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 34069 } } peers { permanent_uuid: "f2ecd2c1710f430ebc1f440a87d5022b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 35889 } }
I20260812 00:10:38.661198 6969 leader_election.cc:288] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303), 33252da37d00420a85f3e06462708300 (127.7.85.195:34069)
W20260812 00:10:38.661665 6148 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fcbcb98b58174c5b8fe1473e5c018c5b (127.7.85.194:35303): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
W20260812 00:10:38.661779 6153 leader_election.cc:341] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33252da37d00420a85f3e06462708300 (127.7.85.195:34069): Not found: Tablet not found: ae854022ee324ad684aeecc16a1bfa42
I20260812 00:10:38.661845 6153 leader_election.cc:302] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2ecd2c1710f430ebc1f440a87d5022b; no voters: 33252da37d00420a85f3e06462708300, fcbcb98b58174c5b8fe1473e5c018c5b
I20260812 00:10:38.661924 6969 raft_consensus.cc:2726] T ae854022ee324ad684aeecc16a1bfa42 P f2ecd2c1710f430ebc1f440a87d5022b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:10:38.827595 6970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:38.827324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:51107)
0812 00:10:38.827505 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:38.827539 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
I20260812 00:10:38.888463 6973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:38.888176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:40409)
0812 00:10:38.888352 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:38.888408 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260812 00:10:39.046234 6974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:39.045957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:44491)
0812 00:10:39.046115 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:39.046171 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260812 00:10:39.828552 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:39.828245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:52649)
0812 00:10:39.828443 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:39.828495 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
I20260812 00:10:39.889487 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:39.889128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:54609)
0812 00:10:39.889369 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:39.889432 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":118,"threads_started":1}
I20260812 00:10:40.047261 6978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:40.046959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:48147)
0812 00:10:40.047138 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:40.047191 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260812 00:10:40.829429 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:40.829151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:46351)
0812 00:10:40.829330 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:40.829376 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260812 00:10:40.890717 6984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:40.890419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:46345)
0812 00:10:40.890602 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:40.890660 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260812 00:10:41.048259 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:41.047947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:57605)
0812 00:10:41.048158 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:41.048206 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":88,"threads_started":1}
I20260812 00:10:41.830430 6986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:41.830109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:51743)
0812 00:10:41.830323 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:41.830375 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":85,"threads_started":1}
I20260812 00:10:41.891646 6987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:41.891363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:43807)
0812 00:10:41.891529 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:41.891587 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260812 00:10:42.049466 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:42.048944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:54683)
0812 00:10:42.049097 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:42.049135 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260812 00:10:42.831301 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:42.831016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:54507)
0812 00:10:42.831202 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:42.831248 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260812 00:10:42.892525 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:42.892254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:39767)
0812 00:10:42.892423 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:42.892462 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":73,"threads_started":1}
I20260812 00:10:43.050619 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:43.050328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:55199)
0812 00:10:43.050510 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:43.050564 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260812 00:10:43.832229 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:43.831942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:47423)
0812 00:10:43.832098 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:43.832143 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260812 00:10:43.893370 7002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:43.893088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:39025)
0812 00:10:43.893255 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:43.893317 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":124,"threads_started":1}
I20260812 00:10:44.051570 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:44.051316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:49401)
0812 00:10:44.051454 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:44.051504 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260812 00:10:44.833339 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:44.833050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.193:40377)
0812 00:10:44.833240 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:44.833281 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":69,"threads_started":1}
I20260812 00:10:44.894325 7006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:44.894047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.194:45351)
0812 00:10:44.894188 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:44.894259 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260812 00:10:45.046932 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 6062
I20260812 00:10:45.053232 7007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:45.052825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37003 (local address 127.7.85.195:56297)
0812 00:10:45.053079 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:45.053131 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.254:37003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":61,"threads_started":1}
I20260812 00:10:45.058825 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 6360
I20260812 00:10:45.077250 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 6629
2026-08-12T00:10:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59224 ms)
[----------] 30 tests from SecurityITest (259186 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:10:45 dist-test-slave-z8x9 krb5kdc[7015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:10:45 dist-test-slave-z8x9 krb5kdc[7015](info): set up 2 sockets
Aug 12 00:10:45 dist-test-slave-z8x9 krb5kdc[7015](info): commencing operation
krb5kdc: starting...
W20260812 00:10:46.523013 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:10:46 dist-test-slave-z8x9 krb5kdc[7015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:10:46.689996 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36841
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:46.789229 7039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:46.789491 7039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:46.789520 7039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:46.790876 7039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:46.790956 7039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:46.790975 7039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:46.790989 7039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:46.791002 7039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:46.792807 7039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:46.793306 7039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:46.793674 7039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:46.800985 7046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:46.801085 7045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:46.801005 7039 server_base.cc:1034] running on GCE node
W20260812 00:10:46.801105 7049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:46.802013 7039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:46.802359 7039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:46.803580 7039 hybrid_clock.cc:648] HybridClock initialized: now 1786493446803515 us; error 81 us; skew 500 ppm
Aug 12 00:10:46 dist-test-slave-z8x9 krb5kdc[7015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493446, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:46.807622 7039 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:10:46.808147 7039 webserver.cc:460] Webserver started at http://127.7.85.254:36041/ using document root <none> and password file <none>
I20260812 00:10:46.808398 7039 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:46.808444 7039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:46.808542 7039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:46.809748 7039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "7ce5091a92b0468f92ee7af686a0eb0b"
format_stamp: "Formatted at 2026-08-12 00:10:46 on dist-test-slave-z8x9"
server_key: "90f769a7202aa1b11091083bb637e940"
server_key_iv: "fa199ee8aedeadab38d6f4321475e799"
server_key_version: "encryptionkey@0"
I20260812 00:10:46.810096 7039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "7ce5091a92b0468f92ee7af686a0eb0b"
format_stamp: "Formatted at 2026-08-12 00:10:46 on dist-test-slave-z8x9"
server_key: "90f769a7202aa1b11091083bb637e940"
server_key_iv: "fa199ee8aedeadab38d6f4321475e799"
server_key_version: "encryptionkey@0"
I20260812 00:10:46.811688 7039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:10:46.812638 7059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:46.812958 7039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:46.813081 7039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "7ce5091a92b0468f92ee7af686a0eb0b"
format_stamp: "Formatted at 2026-08-12 00:10:46 on dist-test-slave-z8x9"
server_key: "90f769a7202aa1b11091083bb637e940"
server_key_iv: "fa199ee8aedeadab38d6f4321475e799"
server_key_version: "encryptionkey@0"
I20260812 00:10:46.813241 7039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:46.825443 7039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:46.828459 7039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:46.828616 7039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:46.833144 7039 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:36841
I20260812 00:10:46.833307 7162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:36841 every 8 connection(s)
I20260812 00:10:46.833673 7039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:46.834651 7164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:46.837455 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 7039
I20260812 00:10:46.837916 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:46.838186 7511 external_mini_cluster.cc:1428] Setting key badd438d0a008b9b3abb22119c1dc36a
I20260812 00:10:46.838524 7164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: Bootstrap starting.
I20260812 00:10:46.839691 7164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:46.840301 7164 log.cc:824] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:46.841445 7164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: No bootstrap required, opened a new log
I20260812 00:10:46.844787 7164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36841 } }
I20260812 00:10:46.844997 7164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:46.845042 7164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce5091a92b0468f92ee7af686a0eb0b, State: Initialized, Role: FOLLOWER
I20260812 00:10:46.845214 7164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36841 } }
I20260812 00:10:46.845316 7164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:46.845358 7164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:46.845424 7164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:46.846316 7164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36841 } }
I20260812 00:10:46.846455 7164 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce5091a92b0468f92ee7af686a0eb0b; no voters:
I20260812 00:10:46.846654 7164 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:46.846740 7173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:46.846956 7164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:46.846964 7173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [term 1 LEADER]: Becoming Leader. State: Replica: 7ce5091a92b0468f92ee7af686a0eb0b, State: Running, Role: LEADER
I20260812 00:10:46.847255 7173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36841 } }
Aug 12 00:10:46 dist-test-slave-z8x9 krb5kdc[7015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:46.847702 7174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36841 } } }
I20260812 00:10:46.847795 7174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:46.848706 7175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce5091a92b0468f92ee7af686a0eb0b. Latest consensus state: current_term: 1 leader_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce5091a92b0468f92ee7af686a0eb0b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36841 } } }
I20260812 00:10:46.848830 7175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b [sys.catalog]: This master's current role is: LEADER
W20260812 00:10:46.849025 7202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:10:46.849123 7202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:10:46.849244 7187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:46.849406 7187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:46.851318 7187 catalog_manager.cc:1347] Generated new cluster ID: f95c2074a6c8401b80056bec1e906249
I20260812 00:10:46.851406 7187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:46.853045 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:46.838865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.7.85.254:36841)
0812 00:10:46.839115 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:46.839124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:46.839152 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:10:46.839662 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:46.839669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:46.839720 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:46.839919 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:46.841043 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:46.842634 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:46.844185 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:46.844688 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:46.844809 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:46.848107 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:46.848112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:46.848113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:46.848145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:46.850778 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:46.851306 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:46.851310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:46.851310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:46.851532 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:46.851850 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:46.851851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:46.851851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:46.852168 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:46.852409 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:46.852549 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 00:10:46.852733 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":110,"threads_started":1}
I20260812 00:10:46.863034 7187 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:46.863839 7187 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:46.873241 7187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: Generated new TSK 0
I20260812 00:10:46.873513 7187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:10:46.913668 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36841
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:47.011756 7215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:47.012027 7215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:47.012073 7215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:47.013290 7215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:47.013370 7215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:47.013420 7215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:47.014935 7215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:47.015225 7215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:47.015476 7215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:47.021221 7240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.021287 7223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.021234 7226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:47.021672 7215 server_base.cc:1034] running on GCE node
I20260812 00:10:47.021934 7215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:47.022217 7215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:47.023406 7215 hybrid_clock.cc:648] HybridClock initialized: now 1786493447023400 us; error 39 us; skew 500 ppm
Aug 12 00:10:47 dist-test-slave-z8x9 krb5kdc[7015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:47.026376 7215 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:10:47.026871 7215 webserver.cc:460] Webserver started at http://127.7.85.193:34647/ using document root <none> and password file <none>
I20260812 00:10:47.027117 7215 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:47.027191 7215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:47.027359 7215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:47.028599 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "42407a4a1d694b72acb4e1a432640ab7"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "255a5936556dba43d660d6cd6dbbfcb3"
server_key_iv: "74a1a5915d164064ab7b0f17a953f60e"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.029014 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "42407a4a1d694b72acb4e1a432640ab7"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "255a5936556dba43d660d6cd6dbbfcb3"
server_key_iv: "74a1a5915d164064ab7b0f17a953f60e"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.030865 7215 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 00:10:47.032030 7253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.032380 7215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:47.032507 7215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "42407a4a1d694b72acb4e1a432640ab7"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "255a5936556dba43d660d6cd6dbbfcb3"
server_key_iv: "74a1a5915d164064ab7b0f17a953f60e"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.032598 7215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:47.044198 7215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:47.046896 7215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:47.047083 7215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:47.047387 7215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:47.047796 7215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:47.047857 7215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.047904 7215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:47.047930 7215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.054597 7215 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:43397
I20260812 00:10:47.054652 7461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:43397 every 8 connection(s)
I20260812 00:10:47.054994 7215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:47.061409 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 7215
I20260812 00:10:47.061560 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:10:47.061800 7511 external_mini_cluster.cc:1428] Setting key 0f70731c7f479069fc4afce74791d699
Aug 12 00:10:47 dist-test-slave-z8x9 krb5kdc[7015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:47.067152 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.055739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36841 (local address 127.7.85.193:39043)
0812 00:10:47.055924 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.055976 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.056475 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.056765 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.056769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.057501 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:47.057706 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.057710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.058440 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.058441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.059318 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.059320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.059813 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.059814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.059963 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.060578 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:47.060591 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:47.062584 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:47.064955 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:47.064968 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:47.064972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:47.065415 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:47.065897 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:47.065898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:47.065899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:47.066084 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:47.066499 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:47.066506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:47.066698 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.066806 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.066959 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260812 00:10:47.067207 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.055791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:39043 (local address 127.7.85.254:36841)
0812 00:10:47.055846 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.055847 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.056053 (+ 206us) server_negotiation.cc:407] Connection header received
0812 00:10:47.056572 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.056573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.056584 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.056649 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:47.057805 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.058336 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.059424 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.059644 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.059706 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.062702 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:47.062704 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:47.062704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:47.062713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:47.064825 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.065583 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.065586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.065587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.065755 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.066257 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.066259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.066259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.066416 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:47.066486 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.066910 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.067048 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:10:47.067926 7463 heartbeater.cc:347] Connected to a master server at 127.7.85.254:36841
I20260812 00:10:47.068076 7463 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:47.068281 7463 heartbeater.cc:511] Master 127.7.85.254:36841 requested a full tablet report, sending...
I20260812 00:10:47.068864 7092 ts_manager.cc:194] Registered new tserver with Master: 42407a4a1d694b72acb4e1a432640ab7 (127.7.85.193:43397)
I20260812 00:10:47.069581 7092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:39043
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:10:47.097016 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36841
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:47.185503 7475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:47.185889 7475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:47.185973 7475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:47.187228 7475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:47.187278 7475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:47.187306 7475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:47.188916 7475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:47.189234 7475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:47.189504 7475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:47.195127 7486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:47.195286 7475 server_base.cc:1034] running on GCE node
W20260812 00:10:47.195185 7482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.195379 7484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:47.195609 7475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:47.195940 7475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:47.197242 7475 hybrid_clock.cc:648] HybridClock initialized: now 1786493447197119 us; error 30 us; skew 500 ppm
Aug 12 00:10:47 dist-test-slave-z8x9 krb5kdc[7015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:47.200451 7475 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:10:47.201274 7475 webserver.cc:460] Webserver started at http://127.7.85.194:45069/ using document root <none> and password file <none>
I20260812 00:10:47.201624 7475 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:47.201712 7475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:47.201906 7475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:47.203253 7475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "6eb2650bb35046dd8cc5db378a18104d"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "11f12b30bc88a4a16ccfded8e21878c2"
server_key_iv: "2943861bd7956cdd95b66c0c45548019"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.203749 7475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "6eb2650bb35046dd8cc5db378a18104d"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "11f12b30bc88a4a16ccfded8e21878c2"
server_key_iv: "2943861bd7956cdd95b66c0c45548019"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.205503 7475 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 00:10:47.206509 7499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.206797 7475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:47.206926 7475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "6eb2650bb35046dd8cc5db378a18104d"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "11f12b30bc88a4a16ccfded8e21878c2"
server_key_iv: "2943861bd7956cdd95b66c0c45548019"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.207013 7475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:47.228309 7475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:47.231246 7475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:47.231457 7475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:47.231761 7475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:47.232123 7475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:47.232193 7475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.232244 7475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:47.232278 7475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.239403 7475 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35585
I20260812 00:10:47.239470 7733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35585 every 8 connection(s)
I20260812 00:10:47.239861 7475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:47.243822 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 7475
I20260812 00:10:47.243978 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:10:47.244169 7511 external_mini_cluster.cc:1428] Setting key 3bdb011a96a28e8b46e5f4f2c83252e8
Aug 12 00:10:47 dist-test-slave-z8x9 krb5kdc[7015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:47.252007 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.240673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:38785 (local address 127.7.85.254:36841)
0812 00:10:47.240762 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.240765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.240994 (+ 229us) server_negotiation.cc:407] Connection header received
0812 00:10:47.241633 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.241635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.241646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.241716 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:47.242777 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.243319 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.244596 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.244894 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.244987 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.248437 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:47.248439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:47.248440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:47.248450 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:47.250111 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.250720 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.250721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.250722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.250779 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.251148 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.251149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.251149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.251281 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:47.251354 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.251829 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.251930 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 00:10:47.252308 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.240620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36841 (local address 127.7.85.194:38785)
0812 00:10:47.240836 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.240880 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.241508 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.241771 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.241775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.242462 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:47.242675 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.242680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.243429 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.243430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.244500 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.244504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.245052 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.245054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.245314 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.245924 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:47.245938 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:47.248299 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:47.250213 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:47.250224 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:47.250227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:47.250618 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:47.250865 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:47.250867 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:47.250867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:47.251047 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:47.251374 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:47.251383 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:47.251592 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.251756 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.251982 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":119,"threads_started":1}
I20260812 00:10:47.252911 7734 heartbeater.cc:347] Connected to a master server at 127.7.85.254:36841
I20260812 00:10:47.253078 7734 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:47.253322 7734 heartbeater.cc:511] Master 127.7.85.254:36841 requested a full tablet report, sending...
I20260812 00:10:47.253836 7092 ts_manager.cc:194] Registered new tserver with Master: 6eb2650bb35046dd8cc5db378a18104d (127.7.85.194:35585)
I20260812 00:10:47.254243 7092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:38785
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:10:47.281610 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36841
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:35161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:47.373850 7743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:47.374130 7743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:47.374176 7743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:47.375444 7743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:47.375527 7743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:47.375578 7743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:47.377308 7743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:35161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:36841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:47.377611 7743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:47.377875 7743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:47.383801 7756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.383920 7751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.383931 7753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:47.383973 7743 server_base.cc:1034] running on GCE node
I20260812 00:10:47.384481 7743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:47.384790 7743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:47.385957 7743 hybrid_clock.cc:648] HybridClock initialized: now 1786493447385931 us; error 41 us; skew 500 ppm
Aug 12 00:10:47 dist-test-slave-z8x9 krb5kdc[7015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:47.388801 7743 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:10:47.389400 7743 webserver.cc:460] Webserver started at http://127.7.85.195:42039/ using document root <none> and password file <none>
I20260812 00:10:47.389657 7743 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:47.389729 7743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:47.389882 7743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:47.391031 7743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "9bb784bcacb04769a5e54a88218d0e1c"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "6cc88cbbc8138dfe99de2d67dd084e98"
server_key_iv: "ed26ba7cea055f24d0807533ed969ba3"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.391414 7743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "9bb784bcacb04769a5e54a88218d0e1c"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "6cc88cbbc8138dfe99de2d67dd084e98"
server_key_iv: "ed26ba7cea055f24d0807533ed969ba3"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.392881 7743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:10:47.393924 7770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.394280 7743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:47.394426 7743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "9bb784bcacb04769a5e54a88218d0e1c"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "6cc88cbbc8138dfe99de2d67dd084e98"
server_key_iv: "ed26ba7cea055f24d0807533ed969ba3"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.394518 7743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:47.426371 7743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:47.429054 7743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:47.429271 7743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:47.429545 7743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:47.429947 7743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:47.430013 7743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.430059 7743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:47.430091 7743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.436954 7743 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:46145
I20260812 00:10:47.437023 8004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:46145 every 8 connection(s)
I20260812 00:10:47.437461 7743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:47.438803 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 7743
I20260812 00:10:47.438921 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:10:47.439096 7511 external_mini_cluster.cc:1428] Setting key 46e2a691e239a7d4b3f4074df72264b2
Aug 12 00:10:47 dist-test-slave-z8x9 krb5kdc[7015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:47.449545 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.438415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:55433 (local address 127.7.85.254:36841)
0812 00:10:47.438491 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.438494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.438718 (+ 224us) server_negotiation.cc:407] Connection header received
0812 00:10:47.439443 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.439444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.439453 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.439532 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:47.440793 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.441473 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.442579 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.442778 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.442864 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.445418 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:47.445420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:47.445420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:47.445429 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:47.447336 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.448021 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.448024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.448025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.448214 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.448656 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.448657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.448658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.448904 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:47.448989 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.449380 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.449466 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:10:47.449733 8008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.438361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36841 (local address 127.7.85.195:55433)
0812 00:10:47.438567 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.438627 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.439260 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.439592 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.439598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.440431 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:47.440657 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.440662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.441587 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.441588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.442472 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.442475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.442889 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.442890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.443018 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.443654 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:47.443663 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:47.445296 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:47.447465 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:47.447474 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:47.447476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:47.447829 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:47.448378 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:47.448379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:47.448380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:47.448521 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:47.448996 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:47.449003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:47.449150 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.449326 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.449483 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
I20260812 00:10:47.450441 8006 heartbeater.cc:347] Connected to a master server at 127.7.85.254:36841
I20260812 00:10:47.450577 8006 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:47.450798 8006 heartbeater.cc:511] Master 127.7.85.254:36841 requested a full tablet report, sending...
I20260812 00:10:47.451440 7091 ts_manager.cc:194] Registered new tserver with Master: 9bb784bcacb04769a5e54a88218d0e1c (127.7.85.195:46145)
I20260812 00:10:47.451915 7091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:55433
I20260812 00:10:47.452327 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:47.460944 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.453882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.7.85.254:36841)
0812 00:10:47.453963 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.453965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.454044 (+ 79us) server_negotiation.cc:407] Connection header received
0812 00:10:47.454097 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.454097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.454106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.454165 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:47.454747 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.455237 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.456056 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.456233 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.456287 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.457767 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:47.457769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:47.457769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:47.457776 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:47.459398 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.459845 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.459848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.459848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.459979 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:47.460284 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:47.460285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:47.460285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:47.460495 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:47.460591 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.460790 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.460868 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 00:10:47.462702 7092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:47.464362 7092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:47.471424 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.467316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59446 (local address 127.7.85.194:35585)
0812 00:10:47.467541 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.467545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.467708 (+ 163us) server_negotiation.cc:407] Connection header received
0812 00:10:47.467756 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.467758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.467772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.467846 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:47.468037 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.469218 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.470402 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.470934 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.471032 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.471101 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.471285 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.471330 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":101,"threads_started":1}
I20260812 00:10:47.471483 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.467002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35585 (local address 127.0.0.1:59446)
0812 00:10:47.467551 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.467583 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.467632 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.467932 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.467932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.467934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:47.467970 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.467971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.469324 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.469325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.470304 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.470307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.471044 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.471046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.471135 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.471150 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.471263 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.471324 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":102,"threads_started":1}
I20260812 00:10:47.471675 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.467215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.7.85.195:46145)
0812 00:10:47.467525 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.467530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.467540 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:10:47.467582 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.467585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.467604 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.467976 (+ 372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:47.468052 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.468757 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.470263 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.471006 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.471079 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.471137 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.471532 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.471589 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":93,"threads_started":1}
I20260812 00:10:47.471487 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.466989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:46145 (local address 127.0.0.1:37564)
0812 00:10:47.467387 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.467399 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.467459 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.467723 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.467727 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.467806 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:47.467891 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.467893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.468866 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.468867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.470158 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.470161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.471176 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.471178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.471259 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.471263 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.471312 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.471361 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":4,"thread_start_us":56,"threads_started":1}
I20260812 00:10:47.472016 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.467196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48176 (local address 127.7.85.193:43397)
0812 00:10:47.467430 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.467435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.467447 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:10:47.467567 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.467570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.467586 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.467661 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:47.467968 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.469151 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.470453 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.471191 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.471255 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.471308 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.471868 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.471921 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":93,"threads_started":1}
I20260812 00:10:47.471693 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.466989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:43397 (local address 127.0.0.1:48176)
0812 00:10:47.467313 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.467356 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.467432 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.467756 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.467757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.467808 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:47.467905 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.467907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.469301 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.469302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.470383 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.470386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.471341 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.471342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.471396 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.471399 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.471445 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.471478 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":6,"spinlock_wait_cycles":768,"thread_start_us":111,"threads_started":1}
I20260812 00:10:47.472534 7854 tablet_service.cc:1467] Processing CreateTablet for tablet 666d447b6ad143159f817e2e09cd738b (DEFAULT_TABLE table=test-table [id=e5a6b42f7c284cc380f566fc153693b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:47.472908 7854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666d447b6ad143159f817e2e09cd738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:47.474146 7344 tablet_service.cc:1467] Processing CreateTablet for tablet 666d447b6ad143159f817e2e09cd738b (DEFAULT_TABLE table=test-table [id=e5a6b42f7c284cc380f566fc153693b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:47.474255 7603 tablet_service.cc:1467] Processing CreateTablet for tablet 666d447b6ad143159f817e2e09cd738b (DEFAULT_TABLE table=test-table [id=e5a6b42f7c284cc380f566fc153693b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:47.474530 7344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666d447b6ad143159f817e2e09cd738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:47.474651 7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 666d447b6ad143159f817e2e09cd738b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:47.476418 8038 tablet_bootstrap.cc:492] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Bootstrap starting.
I20260812 00:10:47.477293 8038 tablet_bootstrap.cc:654] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:47.477800 8039 tablet_bootstrap.cc:492] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Bootstrap starting.
I20260812 00:10:47.477947 8038 log.cc:824] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:47.478134 8041 tablet_bootstrap.cc:492] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: Bootstrap starting.
I20260812 00:10:47.478725 8039 tablet_bootstrap.cc:654] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:47.478799 8038 tablet_bootstrap.cc:492] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: No bootstrap required, opened a new log
I20260812 00:10:47.478858 8041 tablet_bootstrap.cc:654] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:47.478883 8038 ts_tablet_manager.cc:1397] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 00:10:47.479275 8039 log.cc:824] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:47.479499 8041 log.cc:824] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:47.480604 8039 tablet_bootstrap.cc:492] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: No bootstrap required, opened a new log
I20260812 00:10:47.480703 8039 ts_tablet_manager.cc:1397] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:10:47.481386 8038 raft_consensus.cc:348] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.481595 8038 raft_consensus.cc:374] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:47.481640 8038 raft_consensus.cc:729] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb784bcacb04769a5e54a88218d0e1c, State: Initialized, Role: FOLLOWER
I20260812 00:10:47.481645 8041 tablet_bootstrap.cc:492] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: No bootstrap required, opened a new log
I20260812 00:10:47.481714 8041 ts_tablet_manager.cc:1397] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 00:10:47.481777 8038 consensus_queue.cc:260] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.482082 8038 ts_tablet_manager.cc:1428] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 00:10:47.482115 8006 heartbeater.cc:503] Master 127.7.85.254:36841 was elected leader, sending a full tablet report...
I20260812 00:10:47.482844 8039 raft_consensus.cc:348] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.483042 8039 raft_consensus.cc:374] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:47.483095 8039 raft_consensus.cc:729] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb2650bb35046dd8cc5db378a18104d, State: Initialized, Role: FOLLOWER
I20260812 00:10:47.483208 8039 consensus_queue.cc:260] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.483537 8039 ts_tablet_manager.cc:1428] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:47.483593 7734 heartbeater.cc:503] Master 127.7.85.254:36841 was elected leader, sending a full tablet report...
I20260812 00:10:47.484251 8049 raft_consensus.cc:482] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:47.484215 8041 raft_consensus.cc:348] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.484355 8049 raft_consensus.cc:504] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.484431 8041 raft_consensus.cc:374] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:47.484486 8041 raft_consensus.cc:729] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42407a4a1d694b72acb4e1a432640ab7, State: Initialized, Role: FOLLOWER
I20260812 00:10:47.484611 8041 consensus_queue.cc:260] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.484711 8049 leader_election.cc:288] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42407a4a1d694b72acb4e1a432640ab7 (127.7.85.193:43397), 6eb2650bb35046dd8cc5db378a18104d (127.7.85.194:35585)
I20260812 00:10:47.484934 8041 ts_tablet_manager.cc:1428] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:47.485074 7463 heartbeater.cc:503] Master 127.7.85.254:36841 was elected leader, sending a full tablet report...
I20260812 00:10:47.487876 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.485062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:37463 (local address 127.7.85.193:43397)
0812 00:10:47.485107 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.485109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.485116 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:10:47.485137 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.485138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.485146 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.485304 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:47.485461 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.486218 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.487109 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.487503 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.487594 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.487604 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.487774 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.487814 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 00:10:47.487877 8008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.484836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:43397 (local address 127.7.85.195:37463)
0812 00:10:47.484890 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.484904 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.484944 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.485313 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.485314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.485316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:47.485393 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.485395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.486288 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.486289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.487013 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.487015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.487590 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.487591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.487672 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.487676 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.487760 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.487803 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 00:10:47.488332 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.484852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35585 (local address 127.7.85.195:57499)
0812 00:10:47.485018 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.485332 (+ 314us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:47.485398 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:47.485761 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:47.485762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:47.485764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:47.485830 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.485831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.486624 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.486625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.487379 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:47.487381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.488042 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.488044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:47.488128 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.488131 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:47.488218 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 00:10:47.488258 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260812 00:10:47.488339 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.485112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:57499 (local address 127.7.85.194:35585)
0812 00:10:47.485150 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.485152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.485625 (+ 473us) server_negotiation.cc:407] Connection header received
0812 00:10:47.485649 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.485650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.485659 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.485761 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:47.485904 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.486533 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.487488 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.487969 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.488045 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.488055 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.488225 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.488271 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:10:47.488341 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666d447b6ad143159f817e2e09cd738b" candidate_uuid: "9bb784bcacb04769a5e54a88218d0e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42407a4a1d694b72acb4e1a432640ab7" is_pre_election: true
I20260812 00:10:47.488478 7379 raft_consensus.cc:2454] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb784bcacb04769a5e54a88218d0e1c in term 0.
I20260812 00:10:47.488724 7773 leader_election.cc:302] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42407a4a1d694b72acb4e1a432640ab7, 9bb784bcacb04769a5e54a88218d0e1c; no voters:
I20260812 00:10:47.488878 8049 raft_consensus.cc:2781] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:47.488969 8049 raft_consensus.cc:482] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:47.489003 8049 raft_consensus.cc:3037] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:47.488953 7645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666d447b6ad143159f817e2e09cd738b" candidate_uuid: "9bb784bcacb04769a5e54a88218d0e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb2650bb35046dd8cc5db378a18104d" is_pre_election: true
I20260812 00:10:47.489094 7645 raft_consensus.cc:2454] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb784bcacb04769a5e54a88218d0e1c in term 0.
I20260812 00:10:47.489888 8049 raft_consensus.cc:504] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.490053 8049 leader_election.cc:288] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [CANDIDATE]: Term 1 election: Requested vote from peers 42407a4a1d694b72acb4e1a432640ab7 (127.7.85.193:43397), 6eb2650bb35046dd8cc5db378a18104d (127.7.85.194:35585)
I20260812 00:10:47.490221 7379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666d447b6ad143159f817e2e09cd738b" candidate_uuid: "9bb784bcacb04769a5e54a88218d0e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42407a4a1d694b72acb4e1a432640ab7"
I20260812 00:10:47.490298 7645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "666d447b6ad143159f817e2e09cd738b" candidate_uuid: "9bb784bcacb04769a5e54a88218d0e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb2650bb35046dd8cc5db378a18104d"
I20260812 00:10:47.490340 7379 raft_consensus.cc:3037] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:47.490394 7645 raft_consensus.cc:3037] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:47.491182 7379 raft_consensus.cc:2454] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb784bcacb04769a5e54a88218d0e1c in term 1.
I20260812 00:10:47.491191 7645 raft_consensus.cc:2454] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb784bcacb04769a5e54a88218d0e1c in term 1.
W20260812 00:10:47.491400 7735 tablet.cc:2369] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:10:47.491407 7774 leader_election.cc:302] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eb2650bb35046dd8cc5db378a18104d, 9bb784bcacb04769a5e54a88218d0e1c; no voters:
I20260812 00:10:47.491590 8049 raft_consensus.cc:2781] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:47.491797 8049 raft_consensus.cc:686] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 1 LEADER]: Becoming Leader. State: Replica: 9bb784bcacb04769a5e54a88218d0e1c, State: Running, Role: LEADER
I20260812 00:10:47.491904 8049 consensus_queue.cc:237] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } }
I20260812 00:10:47.492974 7092 catalog_manager.cc:5495] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bb784bcacb04769a5e54a88218d0e1c (127.7.85.195). New cstate: current_term: 1 leader_uuid: "9bb784bcacb04769a5e54a88218d0e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bb784bcacb04769a5e54a88218d0e1c" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 } health_report { overall_health: UNKNOWN } } }
I20260812 00:10:47.500758 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.497114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.7.85.195:46145)
0812 00:10:47.497251 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.497253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.497261 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:10:47.497411 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.497411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.497431 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.497493 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:47.497780 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.498320 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.499148 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.499395 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.499566 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.500164 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:47.500295 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:47.500376 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.500571 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.500623 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:10:47.503597 7379 raft_consensus.cc:1261] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 1 FOLLOWER]: Refusing update from remote peer 9bb784bcacb04769a5e54a88218d0e1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:47.503644 7645 raft_consensus.cc:1261] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 1 FOLLOWER]: Refusing update from remote peer 9bb784bcacb04769a5e54a88218d0e1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:47.504136 8060 consensus_queue.cc:1035] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "42407a4a1d694b72acb4e1a432640ab7" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 43397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:47.504436 8049 consensus_queue.cc:1035] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb2650bb35046dd8cc5db378a18104d" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:47.507993 8078 mvcc.cc:205] Tried to move back new op lower bound from 7317477160969830400 to 7317477160927653888. Current Snapshot: MvccSnapshot[applied={T|T < 7317477160969830400 or (T in {7317477160969830400})}]
I20260812 00:10:47.508168 8070 mvcc.cc:205] Tried to move back new op lower bound from 7317477160969830400 to 7317477160927653888. Current Snapshot: MvccSnapshot[applied={T|T < 7317477160969830400}]
I20260812 00:10:47.510298 7092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:47.510457 7092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:47.511559 7092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ce5091a92b0468f92ee7af686a0eb0b: Sending DeleteTablet for 3 replicas of tablet 666d447b6ad143159f817e2e09cd738b
I20260812 00:10:47.511960 7854 tablet_service.cc:1514] Processing DeleteTablet for tablet 666d447b6ad143159f817e2e09cd738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:47 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:37564
I20260812 00:10:47.511996 7603 tablet_service.cc:1514] Processing DeleteTablet for tablet 666d447b6ad143159f817e2e09cd738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:47 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:59446
I20260812 00:10:47.512084 7344 tablet_service.cc:1514] Processing DeleteTablet for tablet 666d447b6ad143159f817e2e09cd738b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:47 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:48176
I20260812 00:10:47.512226 8089 tablet_replica.cc:330] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: stopping tablet replica
I20260812 00:10:47.512336 8089 raft_consensus.cc:2229] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:10:47.512485 8089 raft_consensus.cc:2258] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:47.512497 8095 tablet_replica.cc:330] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7: stopping tablet replica
I20260812 00:10:47.512599 8095 raft_consensus.cc:2229] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:47.512707 8095 raft_consensus.cc:2258] T 666d447b6ad143159f817e2e09cd738b P 42407a4a1d694b72acb4e1a432640ab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:47.512784 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 7215
I20260812 00:10:47.521371 8093 tablet_replica.cc:330] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: stopping tablet replica
I20260812 00:10:47.521623 8093 raft_consensus.cc:2229] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:47.521826 8093 raft_consensus.cc:2258] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:47.536602 8089 ts_tablet_manager.cc:1905] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:47.538754 8089 ts_tablet_manager.cc:1918] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:47.538848 8089 log.cc:1196] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/666d447b6ad143159f817e2e09cd738b
I20260812 00:10:47.539101 8089 ts_tablet_manager.cc:1939] T 666d447b6ad143159f817e2e09cd738b P 9bb784bcacb04769a5e54a88218d0e1c: Deleting consensus metadata
I20260812 00:10:47.543730 8093 ts_tablet_manager.cc:1905] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:47.555545 7060 catalog_manager.cc:4841] TS 9bb784bcacb04769a5e54a88218d0e1c (127.7.85.195:46145): tablet 666d447b6ad143159f817e2e09cd738b (table test-table [id=e5a6b42f7c284cc380f566fc153693b8]) successfully deleted
I20260812 00:10:47.555627 8093 ts_tablet_manager.cc:1918] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:47.555927 8093 log.cc:1196] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/666d447b6ad143159f817e2e09cd738b
I20260812 00:10:47.556336 8093 ts_tablet_manager.cc:1939] T 666d447b6ad143159f817e2e09cd738b P 6eb2650bb35046dd8cc5db378a18104d: Deleting consensus metadata
I20260812 00:10:47.556965 7064 catalog_manager.cc:4841] TS 6eb2650bb35046dd8cc5db378a18104d (127.7.85.194:35585): tablet 666d447b6ad143159f817e2e09cd738b (table test-table [id=e5a6b42f7c284cc380f566fc153693b8]) successfully deleted
W20260812 00:10:47.560580 7062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:43397 (error 108)
I20260812 00:10:47.560812 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 7475
I20260812 00:10:47.589998 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.588480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:43397 (local address 127.0.0.1:48184)
0812 00:10:47.589856 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:47.589936 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:43397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1344}
W20260812 00:10:47.590241 7062 catalog_manager.cc:4584] TS 42407a4a1d694b72acb4e1a432640ab7 (127.7.85.193:43397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 666d447b6ad143159f817e2e09cd738b: Network error: Client connection negotiation failed: client connection to 127.7.85.193:43397: connect: Connection refused (error 111)
I20260812 00:10:47.596988 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 7743
I20260812 00:10:47.616555 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 7039
2026-08-12T00:10:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2524 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T00:10:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:47Z Disabled control of system clock
I20260812 00:10:47.660420 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39799
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:39799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:47.751916 8115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:47.752151 8115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:47.752183 8115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:47.753364 8115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:47.753410 8115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:47.753423 8115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:47.753436 8115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:47.753449 8115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:47.754837 8115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:39799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:39799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:47.755069 8115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:47.755340 8115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:47.760566 8129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.760933 8130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.760936 8133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:47.761109 8115 server_base.cc:1034] running on GCE node
I20260812 00:10:47.761404 8115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:47.761724 8115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:47.762912 8115 hybrid_clock.cc:648] HybridClock initialized: now 1786493447762880 us; error 53 us; skew 500 ppm
I20260812 00:10:47.764356 8115 webserver.cc:460] Webserver started at http://127.7.85.254:37389/ using document root <none> and password file <none>
I20260812 00:10:47.764647 8115 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:47.764750 8115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:47.764926 8115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:47.766193 8115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "55ae487817d34625959d048596be8922"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "d3b4b23a162616d46d9a948143a5b647"
server_key_iv: "0975403e1146f74eb01c29c6c44c9171"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.766572 8115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "55ae487817d34625959d048596be8922"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "d3b4b23a162616d46d9a948143a5b647"
server_key_iv: "0975403e1146f74eb01c29c6c44c9171"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.767980 8115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:10:47.768867 8147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.769131 8115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:47.769277 8115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "55ae487817d34625959d048596be8922"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "d3b4b23a162616d46d9a948143a5b647"
server_key_iv: "0975403e1146f74eb01c29c6c44c9171"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.769374 8115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:47.786707 8115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:47.787120 8115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:47.787295 8115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:47.791918 8115 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:39799
I20260812 00:10:47.791985 8251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:39799 every 8 connection(s)
I20260812 00:10:47.792376 8115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:47.793639 8253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:47.797256 8253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: Bootstrap starting.
I20260812 00:10:47.797454 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8115
I20260812 00:10:47.797574 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:47.797833 7511 external_mini_cluster.cc:1428] Setting key f99e98103c0c3cfe47b0beab698f9c6d
I20260812 00:10:47.798264 8253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:47.798771 8253 log.cc:824] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:47.799733 8253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: No bootstrap required, opened a new log
I20260812 00:10:47.802512 8253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ae487817d34625959d048596be8922" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39799 } }
I20260812 00:10:47.802699 8253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:47.802726 8253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55ae487817d34625959d048596be8922, State: Initialized, Role: FOLLOWER
I20260812 00:10:47.802927 8253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ae487817d34625959d048596be8922" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39799 } }
I20260812 00:10:47.803017 8253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:47.803045 8253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:47.803124 8253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:47.803882 8253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ae487817d34625959d048596be8922" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39799 } }
I20260812 00:10:47.804039 8253 leader_election.cc:302] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55ae487817d34625959d048596be8922; no voters:
I20260812 00:10:47.804313 8253 leader_election.cc:288] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:47.804352 8260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:47.804479 8260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [term 1 LEADER]: Becoming Leader. State: Replica: 55ae487817d34625959d048596be8922, State: Running, Role: LEADER
I20260812 00:10:47.804581 8260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ae487817d34625959d048596be8922" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39799 } }
I20260812 00:10:47.804672 8253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:47.804993 8260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55ae487817d34625959d048596be8922. Latest consensus state: current_term: 1 leader_uuid: "55ae487817d34625959d048596be8922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ae487817d34625959d048596be8922" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39799 } } }
I20260812 00:10:47.805001 8261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55ae487817d34625959d048596be8922" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55ae487817d34625959d048596be8922" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 39799 } } }
I20260812 00:10:47.805091 8260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:47.805126 8261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:47.805442 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:47.798481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.7.85.254:39799)
0812 00:10:47.798758 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:47.798765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:47.798792 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:10:47.799412 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:47.799420 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:47.799483 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:47.799610 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:47.799933 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.800886 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.802428 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:47.803024 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:47.803210 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:47.804010 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:47.804015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:47.804015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:47.804277 (+ 262us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:47.804318 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:47.804337 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:47.804705 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:47.804826 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 00:10:47.804931 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":144,"threads_started":1}
W20260812 00:10:47.806896 8295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:10:47.806967 8295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:10:47.807057 8276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:47.807271 8276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:47.809099 8276 catalog_manager.cc:1347] Generated new cluster ID: f14b858b63d848b9bec33da0d63e3d38
I20260812 00:10:47.809218 8276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:47.831765 8276 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:47.832546 8276 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:47.841802 8276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: Generated new TSK 0
I20260812 00:10:47.842065 8276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:10:47.855423 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39799
--builtin_ntp_servers=127.7.85.212:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:47.947976 8298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:47.948266 8298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:47.948312 8298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:47.949685 8298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:47.949733 8298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:47.949761 8298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:47.951195 8298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:47.951476 8298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:47.951823 8298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:47.958029 8309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.958058 8306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:47.958058 8312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:47.958652 8298 server_base.cc:1034] running on GCE node
I20260812 00:10:47.958899 8298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:47.959226 8298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:47.960417 8298 hybrid_clock.cc:648] HybridClock initialized: now 1786493447960414 us; error 44 us; skew 500 ppm
I20260812 00:10:47.962570 8298 webserver.cc:460] Webserver started at http://127.7.85.193:37259/ using document root <none> and password file <none>
I20260812 00:10:47.962883 8298 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:47.962951 8298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:47.963094 8298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:47.964942 8298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "2bf3872de637455ea578c6db1194d4ca"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "5bcf854ce1dca393e406949db618658a"
server_key_iv: "9ab86cec46b2f9317951470e3e26acec"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.965569 8298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "2bf3872de637455ea578c6db1194d4ca"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "5bcf854ce1dca393e406949db618658a"
server_key_iv: "9ab86cec46b2f9317951470e3e26acec"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.967556 8298 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:10:47.973886 8321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.974155 8298 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260812 00:10:47.974298 8298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "2bf3872de637455ea578c6db1194d4ca"
format_stamp: "Formatted at 2026-08-12 00:10:47 on dist-test-slave-z8x9"
server_key: "5bcf854ce1dca393e406949db618658a"
server_key_iv: "9ab86cec46b2f9317951470e3e26acec"
server_key_version: "encryptionkey@0"
I20260812 00:10:47.974418 8298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:47.991338 8298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:47.991719 8298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:47.991895 8298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:47.992166 8298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:47.992543 8298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:47.992607 8298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.992654 8298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:47.992678 8298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:47.999739 8298 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:38711
I20260812 00:10:47.999788 8547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:38711 every 8 connection(s)
I20260812 00:10:48.000156 8298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:48.004130 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8298
I20260812 00:10:48.004298 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:10:48.004540 7511 external_mini_cluster.cc:1428] Setting key 71e5af66cbf689b9ce2cbeb79c324fa0
I20260812 00:10:48.005767 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39799
--builtin_ntp_servers=127.7.85.212:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:10:48.006659 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.001026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:44117 (local address 127.7.85.254:39799)
0812 00:10:48.001106 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.001109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.001482 (+ 373us) server_negotiation.cc:407] Connection header received
0812 00:10:48.002054 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.002055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.002073 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.002144 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:48.003006 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.003526 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.004730 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.005072 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.005127 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.006247 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:48.006249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:48.006249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:48.006255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:48.006272 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:48.006277 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:48.006338 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.006481 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.006509 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 00:10:48.006884 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.000996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39799 (local address 127.7.85.193:44117)
0812 00:10:48.001317 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.001363 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.001873 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.002282 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.002286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.002295 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:48.002902 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.002906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.003633 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.003634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.004592 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.004595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.005160 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.005161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.005361 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.006068 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:48.006069 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:48.006112 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:48.006146 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:48.006397 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:48.006401 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:48.006403 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.006458 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.006557 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260812 00:10:48.007645 8548 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39799
I20260812 00:10:48.007817 8548 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:48.008052 8548 heartbeater.cc:511] Master 127.7.85.254:39799 requested a full tablet report, sending...
I20260812 00:10:48.008668 8183 ts_manager.cc:194] Registered new tserver with Master: 2bf3872de637455ea578c6db1194d4ca (127.7.85.193:38711)
I20260812 00:10:48.009675 8183 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.193:44117
W20260812 00:10:48.104485 8551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:48.104764 8551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:48.104792 8551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:48.107219 8551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:48.107304 8551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:48.107338 8551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:48.109134 8551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:48.109431 8551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:48.109735 8551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:48.115690 8561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:48.115690 8567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:48.115715 8564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:48.116513 8551 server_base.cc:1034] running on GCE node
I20260812 00:10:48.117087 8551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:48.117478 8551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:48.118965 8551 hybrid_clock.cc:648] HybridClock initialized: now 1786493448118810 us; error 156 us; skew 500 ppm
I20260812 00:10:48.121445 8551 webserver.cc:460] Webserver started at http://127.7.85.194:45719/ using document root <none> and password file <none>
I20260812 00:10:48.121953 8551 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:48.122069 8551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:48.122393 8551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:48.124011 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "d613bf41fb4e4148a497fea58ae9a659"
format_stamp: "Formatted at 2026-08-12 00:10:48 on dist-test-slave-z8x9"
server_key: "29426cc3ecf2f207d563b276f0bfcfcb"
server_key_iv: "46e2dc81bce0d52a054a04be475a5660"
server_key_version: "encryptionkey@0"
I20260812 00:10:48.124838 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "d613bf41fb4e4148a497fea58ae9a659"
format_stamp: "Formatted at 2026-08-12 00:10:48 on dist-test-slave-z8x9"
server_key: "29426cc3ecf2f207d563b276f0bfcfcb"
server_key_iv: "46e2dc81bce0d52a054a04be475a5660"
server_key_version: "encryptionkey@0"
I20260812 00:10:48.127349 8551 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 00:10:48.128475 8576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:48.128783 8551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:48.128917 8551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "d613bf41fb4e4148a497fea58ae9a659"
format_stamp: "Formatted at 2026-08-12 00:10:48 on dist-test-slave-z8x9"
server_key: "29426cc3ecf2f207d563b276f0bfcfcb"
server_key_iv: "46e2dc81bce0d52a054a04be475a5660"
server_key_version: "encryptionkey@0"
I20260812 00:10:48.129029 8551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:48.160282 8551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:48.160679 8551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:48.160862 8551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:48.161130 8551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:48.161803 8551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:48.161979 8551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:48.162027 8551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:48.162058 8551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:48.169195 8551 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:39313
I20260812 00:10:48.169358 8790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:39313 every 8 connection(s)
I20260812 00:10:48.169682 8551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:48.174645 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8551
I20260812 00:10:48.174786 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:10:48.175012 7511 external_mini_cluster.cc:1428] Setting key 036846e9c6d8d82dff49985cda95e5e1
I20260812 00:10:48.176306 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.170714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:53067 (local address 127.7.85.254:39799)
0812 00:10:48.170790 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.170793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.171020 (+ 227us) server_negotiation.cc:407] Connection header received
0812 00:10:48.171539 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.171540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.171552 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.171645 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:48.172480 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.173081 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.174241 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.174519 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.174610 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.175877 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:48.175879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:48.175879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:48.175884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:48.175898 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:48.175902 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:48.175985 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.176107 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.176140 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:10:48.176333 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39799
--builtin_ntp_servers=127.7.85.212:37819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:10:48.176390 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.170634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39799 (local address 127.7.85.194:53067)
0812 00:10:48.170843 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.170887 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.171410 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.171753 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.171757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.171767 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:48.172367 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.172372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.173279 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.173280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.174097 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.174099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.174666 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.174667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.174944 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.175662 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:48.175663 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:48.175705 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:48.175736 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:48.176007 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:48.176014 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:48.176017 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.176079 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.176176 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260812 00:10:48.177014 8792 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39799
I20260812 00:10:48.177158 8792 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:48.177436 8792 heartbeater.cc:511] Master 127.7.85.254:39799 requested a full tablet report, sending...
I20260812 00:10:48.177901 8183 ts_manager.cc:194] Registered new tserver with Master: d613bf41fb4e4148a497fea58ae9a659 (127.7.85.194:39313)
I20260812 00:10:48.178383 8183 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.194:53067
W20260812 00:10:48.265880 8796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:48.266237 8796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:48.266268 8796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:48.267836 8796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:48.267902 8796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:48.267933 8796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:48.269572 8796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:39799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:48.270170 8796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:48.270749 8796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:48.277526 8810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:48.277621 8804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:48.277526 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:48.278277 8796 server_base.cc:1034] running on GCE node
I20260812 00:10:48.278563 8796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:48.278895 8796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:48.280120 8796 hybrid_clock.cc:648] HybridClock initialized: now 1786493448280059 us; error 78 us; skew 500 ppm
I20260812 00:10:48.285794 8796 webserver.cc:460] Webserver started at http://127.7.85.195:38937/ using document root <none> and password file <none>
I20260812 00:10:48.286142 8796 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:48.286232 8796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:48.286408 8796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:48.287604 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "a38d132181ee4929a368f5ff73b655aa"
format_stamp: "Formatted at 2026-08-12 00:10:48 on dist-test-slave-z8x9"
server_key: "a13ebc8ea4d70a71bce7cf8e7c6a0270"
server_key_iv: "ae3f4109aced91cf9d2aac2b4048d12d"
server_key_version: "encryptionkey@0"
I20260812 00:10:48.287994 8796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "a38d132181ee4929a368f5ff73b655aa"
format_stamp: "Formatted at 2026-08-12 00:10:48 on dist-test-slave-z8x9"
server_key: "a13ebc8ea4d70a71bce7cf8e7c6a0270"
server_key_iv: "ae3f4109aced91cf9d2aac2b4048d12d"
server_key_version: "encryptionkey@0"
I20260812 00:10:48.289590 8796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:10:48.290583 8829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:48.291028 8796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:48.291181 8796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "a38d132181ee4929a368f5ff73b655aa"
format_stamp: "Formatted at 2026-08-12 00:10:48 on dist-test-slave-z8x9"
server_key: "a13ebc8ea4d70a71bce7cf8e7c6a0270"
server_key_iv: "ae3f4109aced91cf9d2aac2b4048d12d"
server_key_version: "encryptionkey@0"
I20260812 00:10:48.291276 8796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:48.303875 8796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:48.304296 8796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:48.304468 8796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:48.304741 8796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:48.305161 8796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:48.305264 8796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:48.305300 8796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:48.305313 8796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:48.312333 8796 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:39635
I20260812 00:10:48.312384 9034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:39635 every 8 connection(s)
I20260812 00:10:48.312742 8796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:48.314242 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 8796
I20260812 00:10:48.314337 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:10:48.314625 7511 external_mini_cluster.cc:1428] Setting key 8b1496a48efd205b96cde5a45640285a
I20260812 00:10:48.319785 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.313521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:33831 (local address 127.7.85.254:39799)
0812 00:10:48.313569 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.313571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.313878 (+ 307us) server_negotiation.cc:407] Connection header received
0812 00:10:48.314417 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.314418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.314433 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.314523 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:48.315522 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.316109 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.317546 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.317859 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.317918 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.319303 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:48.319305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:48.319306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:48.319311 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:48.319330 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:48.319334 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:48.319398 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.319637 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.319694 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:10:48.319924 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.313478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:39799 (local address 127.7.85.195:33831)
0812 00:10:48.313683 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.313728 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.314264 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.314592 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.314596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.314605 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:48.315378 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.315386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.316208 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.316209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.317436 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.317440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.317935 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.317936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.318102 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.318998 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:48.318999 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:48.319056 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:48.319100 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:48.319455 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:48.319463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:48.319467 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.319557 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.319671 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260812 00:10:48.320502 9035 heartbeater.cc:347] Connected to a master server at 127.7.85.254:39799
I20260812 00:10:48.320604 9035 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:48.320798 9035 heartbeater.cc:511] Master 127.7.85.254:39799 requested a full tablet report, sending...
I20260812 00:10:48.321288 8183 ts_manager.cc:194] Registered new tserver with Master: a38d132181ee4929a368f5ff73b655aa (127.7.85.195:39635)
I20260812 00:10:48.321652 8183 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.195:33831
I20260812 00:10:48.326974 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:48.331988 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.328537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.7.85.254:39799)
0812 00:10:48.328621 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.328623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.328809 (+ 186us) server_negotiation.cc:407] Connection header received
0812 00:10:48.328834 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.328834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.328844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.328890 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:48.329127 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.329898 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.330668 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.330866 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.330925 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.331707 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:48.331709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:48.331709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:48.331714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:48.331730 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:48.331734 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:48.331781 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.331834 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.331863 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 00:10:48.334631 8183 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:48.336344 8183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:48.343338 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.338892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:39635 (local address 127.0.0.1:54654)
0812 00:10:48.339100 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.339140 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.339211 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.339671 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.339674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.339677 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:48.339759 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.339762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.341090 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.341091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.342243 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.342245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.342979 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.342980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.343073 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.343088 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.343178 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.343237 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":58,"thread_start_us":73,"threads_started":1}
I20260812 00:10:48.343336 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.338983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.7.85.195:39635)
0812 00:10:48.339468 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.339472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.339503 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:10:48.339543 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.339545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.339561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.339841 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:48.339899 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.340992 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.342362 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.342876 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.342979 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.343078 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.343194 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.343237 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":102,"threads_started":1}
I20260812 00:10:48.343531 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.338910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:39313 (local address 127.0.0.1:44148)
0812 00:10:48.339206 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.339217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.339270 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.339715 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.339715 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.339717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:48.339777 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.339780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.341107 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.341108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.342216 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.342219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.343239 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.343240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.343288 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.343292 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.343372 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.343423 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":5,"thread_start_us":85,"threads_started":1}
I20260812 00:10:48.343986 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.339114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38711 (local address 127.0.0.1:33598)
0812 00:10:48.339326 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.339334 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.339369 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.339677 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.339678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.339681 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:48.339789 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.339790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.340718 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.340719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.342004 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.342007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.343732 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.343734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.343803 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.343806 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.343855 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.343901 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":13,"thread_start_us":53,"threads_started":1}
I20260812 00:10:48.344099 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.339009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.7.85.194:39313)
0812 00:10:48.339206 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.339211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.339498 (+ 287us) server_negotiation.cc:407] Connection header received
0812 00:10:48.339545 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.339547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.339560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.339649 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:48.339851 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.341017 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.342351 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.343056 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.343119 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.343195 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.343551 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.343675 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
I20260812 00:10:48.344334 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.339307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.7.85.193:38711)
0812 00:10:48.339476 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.339480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.339492 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:10:48.339541 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.339543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.339560 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.339873 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:48.339940 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.340619 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.342103 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.342872 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.342985 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.343079 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.344191 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.344239 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260812 00:10:48.344587 8914 tablet_service.cc:1467] Processing CreateTablet for tablet a203dfdab4cb4082a40082235ba729a8 (DEFAULT_TABLE table=test-table [id=4c9e9e57aa664ce4a8858e8dd527a565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:48.344991 8914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a203dfdab4cb4082a40082235ba729a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:48.344930 8665 tablet_service.cc:1467] Processing CreateTablet for tablet a203dfdab4cb4082a40082235ba729a8 (DEFAULT_TABLE table=test-table [id=4c9e9e57aa664ce4a8858e8dd527a565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:48.344937 8418 tablet_service.cc:1467] Processing CreateTablet for tablet a203dfdab4cb4082a40082235ba729a8 (DEFAULT_TABLE table=test-table [id=4c9e9e57aa664ce4a8858e8dd527a565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:48.345295 8665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a203dfdab4cb4082a40082235ba729a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:48.345295 8418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a203dfdab4cb4082a40082235ba729a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:48.347833 9063 tablet_bootstrap.cc:492] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Bootstrap starting.
I20260812 00:10:48.348474 9063 tablet_bootstrap.cc:654] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:48.348845 9065 tablet_bootstrap.cc:492] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca: Bootstrap starting.
I20260812 00:10:48.348863 9063 log.cc:824] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:48.349107 9066 tablet_bootstrap.cc:492] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Bootstrap starting.
I20260812 00:10:48.349661 9065 tablet_bootstrap.cc:654] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:48.349722 9063 tablet_bootstrap.cc:492] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: No bootstrap required, opened a new log
I20260812 00:10:48.349822 9066 tablet_bootstrap.cc:654] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:48.349817 9063 ts_tablet_manager.cc:1397] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:10:48.350065 9065 log.cc:824] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:48.350152 9066 log.cc:824] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:48.350863 9065 tablet_bootstrap.cc:492] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca: No bootstrap required, opened a new log
I20260812 00:10:48.350898 9066 tablet_bootstrap.cc:492] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: No bootstrap required, opened a new log
I20260812 00:10:48.350944 9065 ts_tablet_manager.cc:1397] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:10:48.350951 9066 ts_tablet_manager.cc:1397] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:10:48.352097 9063 raft_consensus.cc:348] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.352270 9063 raft_consensus.cc:374] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:48.352319 9063 raft_consensus.cc:729] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38d132181ee4929a368f5ff73b655aa, State: Initialized, Role: FOLLOWER
I20260812 00:10:48.352463 9063 consensus_queue.cc:260] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.352748 9063 ts_tablet_manager.cc:1428] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:48.352788 9035 heartbeater.cc:503] Master 127.7.85.254:39799 was elected leader, sending a full tablet report...
I20260812 00:10:48.353650 9065 raft_consensus.cc:348] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.353822 9065 raft_consensus.cc:374] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:48.353869 9065 raft_consensus.cc:729] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf3872de637455ea578c6db1194d4ca, State: Initialized, Role: FOLLOWER
I20260812 00:10:48.353650 9066 raft_consensus.cc:348] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.353968 9066 raft_consensus.cc:374] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:48.354009 9066 raft_consensus.cc:729] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d613bf41fb4e4148a497fea58ae9a659, State: Initialized, Role: FOLLOWER
I20260812 00:10:48.353988 9065 consensus_queue.cc:260] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.354120 9066 consensus_queue.cc:260] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.354301 8548 heartbeater.cc:503] Master 127.7.85.254:39799 was elected leader, sending a full tablet report...
I20260812 00:10:48.354385 9066 ts_tablet_manager.cc:1428] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 00:10:48.354417 8792 heartbeater.cc:503] Master 127.7.85.254:39799 was elected leader, sending a full tablet report...
I20260812 00:10:48.354566 9065 ts_tablet_manager.cc:1428] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260812 00:10:48.420346 8793 tablet.cc:2369] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:10:48.434063 9072 raft_consensus.cc:482] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:48.434208 9072 raft_consensus.cc:504] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.434602 9072 leader_election.cc:288] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d613bf41fb4e4148a497fea58ae9a659 (127.7.85.194:39313), 2bf3872de637455ea578c6db1194d4ca (127.7.85.193:38711)
I20260812 00:10:48.438413 9039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.434864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:39313 (local address 127.7.85.195:55197)
0812 00:10:48.434960 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.434973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.435082 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.435390 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.435392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.435395 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:48.435482 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.435484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.436355 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.436357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.437321 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.437324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.438103 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.438104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.438182 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.438186 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.438252 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.438298 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":7}
I20260812 00:10:48.438472 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.435150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:55197 (local address 127.7.85.194:39313)
0812 00:10:48.435204 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.435206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.435214 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:10:48.435250 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.435251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.435262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.435328 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:48.435612 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.436178 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.437440 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.437919 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.437977 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.437985 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.438314 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.438357 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 00:10:48.438786 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.435156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:60049 (local address 127.7.85.193:38711)
0812 00:10:48.435288 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.435290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.435369 (+ 79us) server_negotiation.cc:407] Connection header received
0812 00:10:48.435397 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.435398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.435411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.435473 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:48.435817 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.436436 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.437743 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.438339 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.438407 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.438415 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.438657 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.438720 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 00:10:48.438877 8704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a203dfdab4cb4082a40082235ba729a8" candidate_uuid: "a38d132181ee4929a368f5ff73b655aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d613bf41fb4e4148a497fea58ae9a659" is_pre_election: true
I20260812 00:10:48.439059 8704 raft_consensus.cc:2454] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38d132181ee4929a368f5ff73b655aa in term 0.
I20260812 00:10:48.439158 9081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.434992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38711 (local address 127.7.85.195:60049)
0812 00:10:48.435211 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.435222 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:48.435273 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:48.435613 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:48.435614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:48.435617 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:48.435692 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.435694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.436566 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.436567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.437621 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:48.437624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.438455 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.438456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:48.438536 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.438541 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:10:48.438890 (+ 349us) client_negotiation.cc:240] Negotiation successful
0812 00:10:48.438957 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":155,"threads_started":1}
I20260812 00:10:48.439419 8830 leader_election.cc:302] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a38d132181ee4929a368f5ff73b655aa, d613bf41fb4e4148a497fea58ae9a659; no voters:
I20260812 00:10:48.439623 9072 raft_consensus.cc:2781] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:48.439731 9072 raft_consensus.cc:482] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:48.439762 9072 raft_consensus.cc:3037] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:48.439905 8461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a203dfdab4cb4082a40082235ba729a8" candidate_uuid: "a38d132181ee4929a368f5ff73b655aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf3872de637455ea578c6db1194d4ca" is_pre_election: true
I20260812 00:10:48.440073 8461 raft_consensus.cc:2454] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38d132181ee4929a368f5ff73b655aa in term 0.
I20260812 00:10:48.440692 9072 raft_consensus.cc:504] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.441205 9072 leader_election.cc:288] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [CANDIDATE]: Term 1 election: Requested vote from peers d613bf41fb4e4148a497fea58ae9a659 (127.7.85.194:39313), 2bf3872de637455ea578c6db1194d4ca (127.7.85.193:38711)
I20260812 00:10:48.441387 8704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a203dfdab4cb4082a40082235ba729a8" candidate_uuid: "a38d132181ee4929a368f5ff73b655aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d613bf41fb4e4148a497fea58ae9a659"
I20260812 00:10:48.441464 8461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a203dfdab4cb4082a40082235ba729a8" candidate_uuid: "a38d132181ee4929a368f5ff73b655aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bf3872de637455ea578c6db1194d4ca"
I20260812 00:10:48.441498 8704 raft_consensus.cc:3037] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:48.441607 8461 raft_consensus.cc:3037] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:48.442329 8704 raft_consensus.cc:2454] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38d132181ee4929a368f5ff73b655aa in term 1.
I20260812 00:10:48.442415 8461 raft_consensus.cc:2454] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38d132181ee4929a368f5ff73b655aa in term 1.
I20260812 00:10:48.442565 8830 leader_election.cc:302] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a38d132181ee4929a368f5ff73b655aa, d613bf41fb4e4148a497fea58ae9a659; no voters:
I20260812 00:10:48.442662 9072 raft_consensus.cc:2781] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:48.442823 9072 raft_consensus.cc:686] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 1 LEADER]: Becoming Leader. State: Replica: a38d132181ee4929a368f5ff73b655aa, State: Running, Role: LEADER
I20260812 00:10:48.442902 9072 consensus_queue.cc:237] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } }
I20260812 00:10:48.444000 8183 catalog_manager.cc:5495] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa reported cstate change: term changed from 0 to 1, leader changed from <none> to a38d132181ee4929a368f5ff73b655aa (127.7.85.195). New cstate: current_term: 1 leader_uuid: "a38d132181ee4929a368f5ff73b655aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38d132181ee4929a368f5ff73b655aa" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 39635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 } health_report { overall_health: UNKNOWN } } }
I20260812 00:10:48.461267 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.458784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.7.85.195:39635)
0812 00:10:48.458869 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:48.458871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:48.458882 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:10:48.458934 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:48.458935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:48.458960 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:48.459042 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:48.459187 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.459675 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.460429 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:48.460626 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:48.460707 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:48.460869 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:48.460981 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:48.461082 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:48.461101 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 00:10:48.461121 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:10:48.463549 8461 raft_consensus.cc:1261] T a203dfdab4cb4082a40082235ba729a8 P 2bf3872de637455ea578c6db1194d4ca [term 1 FOLLOWER]: Refusing update from remote peer a38d132181ee4929a368f5ff73b655aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:48.463577 8704 raft_consensus.cc:1261] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 1 FOLLOWER]: Refusing update from remote peer a38d132181ee4929a368f5ff73b655aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:48.464013 9082 consensus_queue.cc:1035] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d613bf41fb4e4148a497fea58ae9a659" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:48.464164 9072 consensus_queue.cc:1035] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bf3872de637455ea578c6db1194d4ca" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:48.467325 9088 mvcc.cc:205] Tried to move back new op lower bound from 7317477164903653376 to 7317477164822884352. Current Snapshot: MvccSnapshot[applied={T|T < 7317477164903653376}]
I20260812 00:10:48.469755 8182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:48.469874 8182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:48.470633 8182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55ae487817d34625959d048596be8922: Sending DeleteTablet for 3 replicas of tablet a203dfdab4cb4082a40082235ba729a8
I20260812 00:10:48.471036 8418 tablet_service.cc:1514] Processing DeleteTablet for tablet a203dfdab4cb4082a40082235ba729a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:48 UTC) from {username='slave'} at 127.0.0.1:33598
I20260812 00:10:48.471065 8914 tablet_service.cc:1514] Processing DeleteTablet for tablet a203dfdab4cb4082a40082235ba729a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:48 UTC) from {username='slave'} at 127.0.0.1:54654
I20260812 00:10:48.471381 8665 tablet_service.cc:1514] Processing DeleteTablet for tablet a203dfdab4cb4082a40082235ba729a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:48 UTC) from {username='slave'} at 127.0.0.1:44148
I20260812 00:10:48.471629 9111 tablet_replica.cc:330] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: stopping tablet replica
I20260812 00:10:48.471629 9110 tablet_replica.cc:330] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: stopping tablet replica
I20260812 00:10:48.471745 9111 raft_consensus.cc:2229] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:48.471769 9110 raft_consensus.cc:2229] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:10:48.471815 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8298
I20260812 00:10:48.471917 9110 raft_consensus.cc:2258] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:48.478734 9111 raft_consensus.cc:2258] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:48.479396 9111 ts_tablet_manager.cc:1905] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:48.481417 9111 ts_tablet_manager.cc:1918] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:48.481511 9111 log.cc:1196] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/a203dfdab4cb4082a40082235ba729a8
I20260812 00:10:48.481917 9111 ts_tablet_manager.cc:1939] T a203dfdab4cb4082a40082235ba729a8 P d613bf41fb4e4148a497fea58ae9a659: Deleting consensus metadata
I20260812 00:10:48.482128 9110 ts_tablet_manager.cc:1905] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:48.483140 8148 catalog_manager.cc:4841] TS d613bf41fb4e4148a497fea58ae9a659 (127.7.85.194:39313): tablet a203dfdab4cb4082a40082235ba729a8 (table test-table [id=4c9e9e57aa664ce4a8858e8dd527a565]) successfully deleted
I20260812 00:10:48.483697 9110 ts_tablet_manager.cc:1918] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:48.483816 9110 log.cc:1196] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/a203dfdab4cb4082a40082235ba729a8
I20260812 00:10:48.484076 9110 ts_tablet_manager.cc:1939] T a203dfdab4cb4082a40082235ba729a8 P a38d132181ee4929a368f5ff73b655aa: Deleting consensus metadata
I20260812 00:10:48.484843 8158 catalog_manager.cc:4841] TS a38d132181ee4929a368f5ff73b655aa (127.7.85.195:39635): tablet a203dfdab4cb4082a40082235ba729a8 (table test-table [id=4c9e9e57aa664ce4a8858e8dd527a565]) successfully deleted
W20260812 00:10:48.486159 8148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:38711 (error 108)
I20260812 00:10:48.486778 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:48.486589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38711 (local address 127.0.0.1:33602)
0812 00:10:48.486655 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:48.486731 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:10:48.486811 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8551
W20260812 00:10:48.506508 8148 catalog_manager.cc:4584] TS 2bf3872de637455ea578c6db1194d4ca (127.7.85.193:38711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a203dfdab4cb4082a40082235ba729a8: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38711: connect: Connection refused (error 111)
I20260812 00:10:48.515065 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8796
I20260812 00:10:48.534355 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 8115
2026-08-12T00:10:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (918 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:10:48 dist-test-slave-z8x9 krb5kdc[9116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:10:48 dist-test-slave-z8x9 krb5kdc[9116](info): set up 2 sockets
Aug 12 00:10:48 dist-test-slave-z8x9 krb5kdc[9116](info): commencing operation
krb5kdc: starting...
W20260812 00:10:49.909827 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:10:50.056759 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37513
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:37513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:50.150342 9140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:50.150632 9140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:50.150677 9140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:50.152691 9140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:50.152805 9140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:50.152835 9140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:50.152868 9140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:50.152904 9140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:50.154800 9140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:37513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:37513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:50.155149 9140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:50.155447 9140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:50.161451 9146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.161525 9149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.161451 9147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:50.161849 9140 server_base.cc:1034] running on GCE node
I20260812 00:10:50.162509 9140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:50.163053 9140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:50.164273 9140 hybrid_clock.cc:648] HybridClock initialized: now 1786493450164267 us; error 71 us; skew 500 ppm
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:50.167743 9140 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:10:50.168263 9140 webserver.cc:460] Webserver started at http://127.7.85.254:39429/ using document root <none> and password file <none>
I20260812 00:10:50.168545 9140 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:50.168623 9140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:50.168782 9140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:50.169963 9140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "8a19feb2173947dcbed1aece06ed195d"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "061fb63c8c7c6379871d36dad4db6925"
server_key_iv: "27d93a43d287c2999471e005cf9f4937"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.170354 9140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "8a19feb2173947dcbed1aece06ed195d"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "061fb63c8c7c6379871d36dad4db6925"
server_key_iv: "27d93a43d287c2999471e005cf9f4937"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.171976 9140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:10:50.173089 9166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.173411 9140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:50.173552 9140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "8a19feb2173947dcbed1aece06ed195d"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "061fb63c8c7c6379871d36dad4db6925"
server_key_iv: "27d93a43d287c2999471e005cf9f4937"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.173656 9140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:50.202409 9140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:50.205056 9140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:50.205278 9140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:50.209838 9140 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:37513
I20260812 00:10:50.210004 9264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:37513 every 8 connection(s)
I20260812 00:10:50.210314 9140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:50.211301 9266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:50.213866 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9140
I20260812 00:10:50.213886 9266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: Bootstrap starting.
I20260812 00:10:50.213970 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:50.214221 7511 external_mini_cluster.cc:1428] Setting key 2c359c16a6564953ad371cf0fef1430f
I20260812 00:10:50.215086 9266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:50.215754 9266 log.cc:824] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:50.216959 9266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: No bootstrap required, opened a new log
I20260812 00:10:50.219983 9266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a19feb2173947dcbed1aece06ed195d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37513 } }
I20260812 00:10:50.220183 9266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:50.220235 9266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a19feb2173947dcbed1aece06ed195d, State: Initialized, Role: FOLLOWER
I20260812 00:10:50.220405 9266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a19feb2173947dcbed1aece06ed195d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37513 } }
I20260812 00:10:50.220510 9266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:50.220563 9266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:50.220620 9266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:50.221508 9266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a19feb2173947dcbed1aece06ed195d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37513 } }
I20260812 00:10:50.221634 9266 leader_election.cc:302] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a19feb2173947dcbed1aece06ed195d; no voters:
I20260812 00:10:50.221854 9266 leader_election.cc:288] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:50.222007 9283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:50.222182 9283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [term 1 LEADER]: Becoming Leader. State: Replica: 8a19feb2173947dcbed1aece06ed195d, State: Running, Role: LEADER
I20260812 00:10:50.222247 9266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:50.222270 9283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a19feb2173947dcbed1aece06ed195d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37513 } }
I20260812 00:10:50.222620 9283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a19feb2173947dcbed1aece06ed195d. Latest consensus state: current_term: 1 leader_uuid: "8a19feb2173947dcbed1aece06ed195d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a19feb2173947dcbed1aece06ed195d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37513 } } }
I20260812 00:10:50.222713 9283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:50.222636 9284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a19feb2173947dcbed1aece06ed195d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a19feb2173947dcbed1aece06ed195d" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 37513 } } }
I20260812 00:10:50.222926 9284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d [sys.catalog]: This master's current role is: LEADER
W20260812 00:10:50.224346 9310 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:10:50.224416 9310 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:10:50.224475 9289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:50.224655 9289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:50.226208 9289 catalog_manager.cc:1347] Generated new cluster ID: 48936708af3a47bc9a874b7f70216f5e
I20260812 00:10:50.226261 9289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:50.230406 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.214995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.7.85.254:37513)
0812 00:10:50.215344 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.215358 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.215415 (+ 57us) server_negotiation.cc:407] Connection header received
0812 00:10:50.216558 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.216573 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.216661 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.216856 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:50.217670 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.219502 (+ 1832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.220939 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.221694 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.221831 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.226025 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:50.226030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:50.226031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:50.226045 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:50.228358 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.228803 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.228807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.228808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.228933 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.229156 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.229158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.229158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.229556 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:50.229760 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.229853 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.230139 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":93,"threads_started":1}
I20260812 00:10:50.236698 9289 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:50.237602 9289 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:50.251011 9289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: Generated new TSK 0
I20260812 00:10:50.251260 9289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:10:50.284113 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37513
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:50.372507 9320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:50.372736 9320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:50.372761 9320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:50.374037 9320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:50.374090 9320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:50.374117 9320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:50.375630 9320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:50.375905 9320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:50.376190 9320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:50.381636 9329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.381762 9332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.381651 9335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:50.382161 9320 server_base.cc:1034] running on GCE node
I20260812 00:10:50.382377 9320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:50.382618 9320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:50.383836 9320 hybrid_clock.cc:648] HybridClock initialized: now 1786493450383812 us; error 31 us; skew 500 ppm
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:50.386881 9320 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:10:50.387427 9320 webserver.cc:460] Webserver started at http://127.7.85.193:45297/ using document root <none> and password file <none>
I20260812 00:10:50.387699 9320 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:50.387792 9320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:50.387965 9320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:50.389313 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "cb71f3c5b4984712b62a91d3c187ae5a"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "3e22feea2c231cd37de73b3ac8d1da55"
server_key_iv: "1371e6ff85181495f52ee40ab69cf3c8"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.389760 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "cb71f3c5b4984712b62a91d3c187ae5a"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "3e22feea2c231cd37de73b3ac8d1da55"
server_key_iv: "1371e6ff85181495f52ee40ab69cf3c8"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.391363 9320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:10:50.392383 9347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.392778 9320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:50.392997 9320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "cb71f3c5b4984712b62a91d3c187ae5a"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "3e22feea2c231cd37de73b3ac8d1da55"
server_key_iv: "1371e6ff85181495f52ee40ab69cf3c8"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.393119 9320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:50.434161 9320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:50.437067 9320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:50.437336 9320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:50.437628 9320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:50.438001 9320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:50.438064 9320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.438112 9320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:50.438136 9320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.447372 9320 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:33167
I20260812 00:10:50.447533 9590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:33167 every 8 connection(s)
I20260812 00:10:50.448009 9320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:50.451319 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9320
I20260812 00:10:50.451431 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:10:50.451701 7511 external_mini_cluster.cc:1428] Setting key 1408d4c0060936f957cd1110e2fbf07f
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:50.460366 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.449403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:54601 (local address 127.7.85.254:37513)
0812 00:10:50.449464 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.449467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.449825 (+ 358us) server_negotiation.cc:407] Connection header received
0812 00:10:50.450420 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.450421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.450434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.450512 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:50.451675 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.452358 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.453976 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.454188 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.454255 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.457092 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:50.457094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:50.457095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:50.457103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:50.458760 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.459254 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.459255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.459256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.459324 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.459629 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.459630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.459630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.459772 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:50.459856 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.460166 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.460263 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 00:10:50.460587 9598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.449179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37513 (local address 127.7.85.193:54601)
0812 00:10:50.449587 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.449666 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.450288 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.450604 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.450608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.451264 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:50.451558 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.451563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.452471 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.452472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.453828 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.453831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.454289 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.454291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.454462 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.455212 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:50.455222 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:50.456972 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:50.458879 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:50.458888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:50.458891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:50.459145 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:50.459400 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:50.459401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:50.459401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:50.459542 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:50.459858 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:50.459867 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:50.460022 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.460143 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.460334 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":118,"threads_started":1}
I20260812 00:10:50.461261 9594 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37513
I20260812 00:10:50.461380 9594 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:50.461621 9594 heartbeater.cc:511] Master 127.7.85.254:37513 requested a full tablet report, sending...
I20260812 00:10:50.462435 9197 ts_manager.cc:194] Registered new tserver with Master: cb71f3c5b4984712b62a91d3c187ae5a (127.7.85.193:33167)
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:10:50.463471 9197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:54601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:10:50.490662 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37513
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:50.580327 9603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:50.580562 9603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:50.580590 9603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:50.581784 9603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:50.581826 9603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:50.581854 9603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:50.583343 9603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:50.583562 9603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:50.583760 9603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:50.589682 9618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.589686 9613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.589690 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:50.589836 9603 server_base.cc:1034] running on GCE node
I20260812 00:10:50.590207 9603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:50.590457 9603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:50.591668 9603 hybrid_clock.cc:648] HybridClock initialized: now 1786493450591655 us; error 53 us; skew 500 ppm
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:50.594666 9603 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:10:50.595240 9603 webserver.cc:460] Webserver started at http://127.7.85.194:36383/ using document root <none> and password file <none>
I20260812 00:10:50.595507 9603 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:50.595584 9603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:50.595748 9603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:50.596915 9603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "505d89b0a244486497805e0cb7a89523"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "8af4b0f20d9f7a8b699f6828f68afbe4"
server_key_iv: "d3bc8deeda5b89f8e94f0187139c95fd"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.597455 9603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "505d89b0a244486497805e0cb7a89523"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "8af4b0f20d9f7a8b699f6828f68afbe4"
server_key_iv: "d3bc8deeda5b89f8e94f0187139c95fd"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.599170 9603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 00:10:50.600221 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.600507 9603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:10:50.600629 9603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "505d89b0a244486497805e0cb7a89523"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "8af4b0f20d9f7a8b699f6828f68afbe4"
server_key_iv: "d3bc8deeda5b89f8e94f0187139c95fd"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.600739 9603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:50.614324 9603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:50.617110 9603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:50.617429 9603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:50.617745 9603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:50.618140 9603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:50.618217 9603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.618259 9603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:50.618292 9603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.625720 9603 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:45071
I20260812 00:10:50.625814 9857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:45071 every 8 connection(s)
I20260812 00:10:50.626230 9603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:50.628451 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9603
I20260812 00:10:50.628639 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:10:50.628896 7511 external_mini_cluster.cc:1428] Setting key a0de9ad827b550a143b54202dca0d1ce
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:50.638203 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.627211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:48131 (local address 127.7.85.254:37513)
0812 00:10:50.627312 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.627314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.627585 (+ 271us) server_negotiation.cc:407] Connection header received
0812 00:10:50.628039 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.628040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.628053 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.628125 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:50.629623 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.630142 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.631202 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.631480 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.631563 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.634310 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:50.634313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:50.634313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:50.634322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:50.636192 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.637020 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.637022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.637023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.637103 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.637443 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.637444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.637445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.637586 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:50.637674 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.637995 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.638104 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:10:50.638417 9862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.627179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37513 (local address 127.7.85.194:48131)
0812 00:10:50.627387 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.627433 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.627923 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.628276 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.628280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.629231 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:50.629514 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.629520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.630253 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.630254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.631093 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.631095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.631607 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.631608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.631837 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.632459 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:50.632468 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:50.634194 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:50.636627 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:50.636637 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:50.636640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:50.636914 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:50.637204 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:50.637205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:50.637205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:50.637361 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:50.637663 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:50.637672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:50.637837 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.637953 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.638140 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
I20260812 00:10:50.638985 9859 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37513
I20260812 00:10:50.639091 9859 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:50.639312 9859 heartbeater.cc:511] Master 127.7.85.254:37513 requested a full tablet report, sending...
I20260812 00:10:50.639751 9197 ts_manager.cc:194] Registered new tserver with Master: 505d89b0a244486497805e0cb7a89523 (127.7.85.194:45071)
I20260812 00:10:50.640249 9197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:48131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:10:50.664812 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37513
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:50.755388 9872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:50.755623 9872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:50.755650 9872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:50.756770 9872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:50.756808 9872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:50.756834 9872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:50.758495 9872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:37513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:50.758699 9872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:50.758876 9872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:50.764721 9895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:50.764890 9872 server_base.cc:1034] running on GCE node
W20260812 00:10:50.764750 9892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:50.764986 9889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:50.765298 9872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:50.765578 9872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:50.766762 9872 hybrid_clock.cc:648] HybridClock initialized: now 1786493450766757 us; error 41 us; skew 500 ppm
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:50.769985 9872 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:10:50.770505 9872 webserver.cc:460] Webserver started at http://127.7.85.195:36013/ using document root <none> and password file <none>
I20260812 00:10:50.770762 9872 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:50.770843 9872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:50.771008 9872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:50.772202 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "bddd8080d9eb43c69052726c1661f28a"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "1177733fb6a7e876028e108eb46f1480"
server_key_iv: "e1e097a87c9a12a351bd6b32d281152e"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.772614 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "bddd8080d9eb43c69052726c1661f28a"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "1177733fb6a7e876028e108eb46f1480"
server_key_iv: "e1e097a87c9a12a351bd6b32d281152e"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.774283 9872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 00:10:50.775187 9908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.775406 9872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:50.775514 9872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "bddd8080d9eb43c69052726c1661f28a"
format_stamp: "Formatted at 2026-08-12 00:10:50 on dist-test-slave-z8x9"
server_key: "1177733fb6a7e876028e108eb46f1480"
server_key_iv: "e1e097a87c9a12a351bd6b32d281152e"
server_key_version: "encryptionkey@0"
I20260812 00:10:50.775576 9872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:50.790603 9872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:50.793063 9872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:50.793293 9872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:50.793527 9872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:50.793797 9872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:50.793828 9872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.793851 9872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:50.793867 9872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:50.800645 9872 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:44399
I20260812 00:10:50.800743 10144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:44399 every 8 connection(s)
I20260812 00:10:50.801105 9872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:50.802647 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 9872
I20260812 00:10:50.802752 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:10:50.802932 7511 external_mini_cluster.cc:1428] Setting key 3b5d59159c8dc25c28a43aa49e453eaa
Aug 12 00:10:50 dist-test-slave-z8x9 krb5kdc[9116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:50.812477 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.802004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:42683 (local address 127.7.85.254:37513)
0812 00:10:50.802144 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.802146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.802329 (+ 183us) server_negotiation.cc:407] Connection header received
0812 00:10:50.803131 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.803132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.803147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.803210 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:50.804487 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.805015 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.806230 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.806420 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.806488 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.809094 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:50.809096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:50.809097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:50.809104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:50.810736 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.811172 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.811174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.811174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.811270 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.811795 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.811796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.811797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.811928 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:50.811999 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.812330 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.812406 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 00:10:50.812641 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.801905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:37513 (local address 127.7.85.195:42683)
0812 00:10:50.802173 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.802225 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.802991 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.803293 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.803297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.804179 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:50.804392 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.804397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.805144 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.805146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.806067 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.806070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.806540 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.806541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.806672 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.807308 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:50.807317 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:50.808943 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:50.810827 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:50.810834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:50.810836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:50.811073 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:50.811546 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:50.811547 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:50.811548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:50.811680 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:50.812024 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:50.812031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:50.812174 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.812282 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.812422 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":127,"threads_started":1}
I20260812 00:10:50.813311 10148 heartbeater.cc:347] Connected to a master server at 127.7.85.254:37513
I20260812 00:10:50.813423 10148 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:50.813591 10148 heartbeater.cc:511] Master 127.7.85.254:37513 requested a full tablet report, sending...
I20260812 00:10:50.814029 9197 ts_manager.cc:194] Registered new tserver with Master: bddd8080d9eb43c69052726c1661f28a (127.7.85.195:44399)
I20260812 00:10:50.814620 9197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:42683
I20260812 00:10:50.814652 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:50.824285 9274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.816124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.7.85.254:37513)
0812 00:10:50.816210 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.816212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.816523 (+ 311us) server_negotiation.cc:407] Connection header received
0812 00:10:50.816576 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.816578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.816596 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.816695 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:50.817372 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.817888 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.818657 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.818873 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.818943 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.820755 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:50.820757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:50.820757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:50.820765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:50.822835 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.823271 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.823272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.823272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.823332 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:50.823526 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:50.823526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:50.823526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:50.823664 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:50.823790 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.824080 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.824144 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:10:50.826905 9197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:50.828557 9197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:50.836958 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.831999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.7.85.193:33167)
0812 00:10:50.832217 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.832223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.832734 (+ 511us) server_negotiation.cc:407] Connection header received
0812 00:10:50.832789 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.832792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.832808 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.832886 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:50.833005 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.833845 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.835619 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.836435 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.836569 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.836639 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.836829 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.836873 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260812 00:10:50.836954 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.831835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:45071 (local address 127.0.0.1:53640)
0812 00:10:50.832466 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.832479 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.832568 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.833147 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.833148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.833157 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:50.833235 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.833237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.834421 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.834422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.835524 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.835527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.836649 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.836650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.836721 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.836726 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.836758 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.836833 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260812 00:10:50.836952 10171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.831835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:33167 (local address 127.0.0.1:47538)
0812 00:10:50.832666 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.832676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.832727 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.832904 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.832905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.832907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:50.832943 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.832945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.833974 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.833976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.835499 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.835501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.836573 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.836574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.836676 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.836692 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.836809 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.836862 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":583,"thread_start_us":83,"threads_started":1}
I20260812 00:10:50.837384 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.832361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53640 (local address 127.7.85.194:45071)
0812 00:10:50.832551 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.832556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.832568 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:10:50.833027 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.833030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.833049 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.833134 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:50.833333 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.834299 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.835655 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.836543 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.836889 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.836955 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.836984 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.837307 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
I20260812 00:10:50.837751 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.831835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44399 (local address 127.0.0.1:59266)
0812 00:10:50.832221 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.832269 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.832359 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.832615 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.832619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.832621 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:50.832716 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.832719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.834143 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.834143 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.835496 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.835500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.836701 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.836702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.836759 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.836763 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.836809 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.836842 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260812 00:10:50.837750 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.832024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.7.85.195:44399)
0812 00:10:50.832272 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.832276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.832337 (+ 61us) server_negotiation.cc:407] Connection header received
0812 00:10:50.832480 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.832482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.832499 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.832595 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:50.832830 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.834023 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.835602 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.836505 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.836590 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.836657 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.836994 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.837039 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":134,"threads_started":1}
I20260812 00:10:50.838238 9719 tablet_service.cc:1467] Processing CreateTablet for tablet 8144239d500d4a03b360d9c6c866a41e (DEFAULT_TABLE table=test-table [id=1e9d899a33dc492baa7b04517680e0a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:50.838238 9444 tablet_service.cc:1467] Processing CreateTablet for tablet 8144239d500d4a03b360d9c6c866a41e (DEFAULT_TABLE table=test-table [id=1e9d899a33dc492baa7b04517680e0a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:50.838488 9998 tablet_service.cc:1467] Processing CreateTablet for tablet 8144239d500d4a03b360d9c6c866a41e (DEFAULT_TABLE table=test-table [id=1e9d899a33dc492baa7b04517680e0a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:50.838639 9719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8144239d500d4a03b360d9c6c866a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:50.838639 9444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8144239d500d4a03b360d9c6c866a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:50.838868 9998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8144239d500d4a03b360d9c6c866a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:50.841702 10173 tablet_bootstrap.cc:492] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Bootstrap starting.
I20260812 00:10:50.842314 10176 tablet_bootstrap.cc:492] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Bootstrap starting.
I20260812 00:10:50.842542 10173 tablet_bootstrap.cc:654] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:50.843036 10176 tablet_bootstrap.cc:654] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:50.843159 10173 log.cc:824] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:50.842116 10178 tablet_bootstrap.cc:492] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a: Bootstrap starting.
I20260812 00:10:50.843617 10176 log.cc:824] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:50.844092 10173 tablet_bootstrap.cc:492] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: No bootstrap required, opened a new log
I20260812 00:10:50.844156 10178 tablet_bootstrap.cc:654] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:50.844174 10173 ts_tablet_manager.cc:1397] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:10:50.844461 10176 tablet_bootstrap.cc:492] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: No bootstrap required, opened a new log
I20260812 00:10:50.844544 10176 ts_tablet_manager.cc:1397] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:10:50.844899 10178 log.cc:824] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:50.845746 10178 tablet_bootstrap.cc:492] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a: No bootstrap required, opened a new log
I20260812 00:10:50.845819 10178 ts_tablet_manager.cc:1397] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 00:10:50.846207 10173 raft_consensus.cc:348] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.846390 10173 raft_consensus.cc:374] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:50.846438 10173 raft_consensus.cc:729] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddd8080d9eb43c69052726c1661f28a, State: Initialized, Role: FOLLOWER
I20260812 00:10:50.846560 10173 consensus_queue.cc:260] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.846724 10176 raft_consensus.cc:348] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.846894 10176 raft_consensus.cc:374] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:50.846877 10173 ts_tablet_manager.cc:1428] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:50.846931 10176 raft_consensus.cc:729] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 505d89b0a244486497805e0cb7a89523, State: Initialized, Role: FOLLOWER
I20260812 00:10:50.847069 10148 heartbeater.cc:503] Master 127.7.85.254:37513 was elected leader, sending a full tablet report...
I20260812 00:10:50.847060 10176 consensus_queue.cc:260] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.847326 10176 ts_tablet_manager.cc:1428] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:50.847348 9859 heartbeater.cc:503] Master 127.7.85.254:37513 was elected leader, sending a full tablet report...
I20260812 00:10:50.848016 10178 raft_consensus.cc:348] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.848207 10178 raft_consensus.cc:374] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:50.848258 10178 raft_consensus.cc:729] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb71f3c5b4984712b62a91d3c187ae5a, State: Initialized, Role: FOLLOWER
I20260812 00:10:50.848371 10178 consensus_queue.cc:260] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.848628 10178 ts_tablet_manager.cc:1428] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:50.848694 9594 heartbeater.cc:503] Master 127.7.85.254:37513 was elected leader, sending a full tablet report...
W20260812 00:10:50.876924 9861 tablet.cc:2369] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:10:50.898901 10191 raft_consensus.cc:482] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:50.899046 10191 raft_consensus.cc:504] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.899569 10191 leader_election.cc:288] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 505d89b0a244486497805e0cb7a89523 (127.7.85.194:45071), bddd8080d9eb43c69052726c1661f28a (127.7.85.195:44399)
I20260812 00:10:50.902884 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.899943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:56415 (local address 127.7.85.194:45071)
0812 00:10:50.900008 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.900010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.900061 (+ 51us) server_negotiation.cc:407] Connection header received
0812 00:10:50.900157 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.900158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.900171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.900229 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:50.900447 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.900912 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.902090 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.902492 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.902590 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.902601 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.902756 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.902804 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:10:50.902904 9598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.899816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:44399 (local address 127.7.85.193:53683)
0812 00:10:50.899881 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.899894 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.899951 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.900187 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.900188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.900190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:50.900279 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.900282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.900977 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.900979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.901981 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.901984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.902590 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.902591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.902696 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.902699 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.902780 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.902824 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":6}
I20260812 00:10:50.902942 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.899987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:53683 (local address 127.7.85.195:44399)
0812 00:10:50.900042 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.900045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.900055 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:10:50.900087 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.900088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.900098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.900166 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:50.900376 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.900881 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.902102 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.902492 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.902590 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.902601 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.902823 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.902873 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 00:10:50.902907 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.899835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:45071 (local address 127.7.85.193:56415)
0812 00:10:50.899986 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:50.899998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:50.900070 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:50.900260 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:50.900261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:50.900262 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:50.900356 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.900358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.901032 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.901033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.901986 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:50.901989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.902591 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.902592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:50.902675 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.902679 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:50.902725 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 00:10:50.902788 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":15,"thread_start_us":69,"threads_started":1}
I20260812 00:10:50.903420 9765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8144239d500d4a03b360d9c6c866a41e" candidate_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505d89b0a244486497805e0cb7a89523" is_pre_election: true
I20260812 00:10:50.903457 10036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8144239d500d4a03b360d9c6c866a41e" candidate_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bddd8080d9eb43c69052726c1661f28a" is_pre_election: true
I20260812 00:10:50.903602 9765 raft_consensus.cc:2454] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb71f3c5b4984712b62a91d3c187ae5a in term 0.
I20260812 00:10:50.903678 10036 raft_consensus.cc:2454] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb71f3c5b4984712b62a91d3c187ae5a in term 0.
I20260812 00:10:50.903884 9349 leader_election.cc:302] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505d89b0a244486497805e0cb7a89523, cb71f3c5b4984712b62a91d3c187ae5a; no voters:
I20260812 00:10:50.904047 10191 raft_consensus.cc:2781] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:50.904110 10191 raft_consensus.cc:482] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:50.904143 10191 raft_consensus.cc:3037] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:50.905048 10191 raft_consensus.cc:504] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.905259 10191 leader_election.cc:288] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [CANDIDATE]: Term 1 election: Requested vote from peers 505d89b0a244486497805e0cb7a89523 (127.7.85.194:45071), bddd8080d9eb43c69052726c1661f28a (127.7.85.195:44399)
I20260812 00:10:50.905485 9765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8144239d500d4a03b360d9c6c866a41e" candidate_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "505d89b0a244486497805e0cb7a89523"
I20260812 00:10:50.905485 10036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8144239d500d4a03b360d9c6c866a41e" candidate_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bddd8080d9eb43c69052726c1661f28a"
I20260812 00:10:50.905591 9765 raft_consensus.cc:3037] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:50.905591 10036 raft_consensus.cc:3037] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:50.906589 9765 raft_consensus.cc:2454] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb71f3c5b4984712b62a91d3c187ae5a in term 1.
I20260812 00:10:50.906591 10036 raft_consensus.cc:2454] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb71f3c5b4984712b62a91d3c187ae5a in term 1.
I20260812 00:10:50.906848 9349 leader_election.cc:302] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 505d89b0a244486497805e0cb7a89523, cb71f3c5b4984712b62a91d3c187ae5a; no voters:
I20260812 00:10:50.906945 10191 raft_consensus.cc:2781] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:50.907121 10191 raft_consensus.cc:686] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [term 1 LEADER]: Becoming Leader. State: Replica: cb71f3c5b4984712b62a91d3c187ae5a, State: Running, Role: LEADER
I20260812 00:10:50.907213 10191 consensus_queue.cc:237] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } }
I20260812 00:10:50.908213 9197 catalog_manager.cc:5495] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a reported cstate change: term changed from 0 to 1, leader changed from <none> to cb71f3c5b4984712b62a91d3c187ae5a (127.7.85.193). New cstate: current_term: 1 leader_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb71f3c5b4984712b62a91d3c187ae5a" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 33167 } health_report { overall_health: HEALTHY } } }
I20260812 00:10:50.933879 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:50.930773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.7.85.193:33167)
0812 00:10:50.930905 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:50.930907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:50.930918 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:10:50.930962 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:50.930963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:50.930985 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:50.931061 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:50.931315 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.931890 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.932896 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:50.933125 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:50.933255 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:50.933478 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:50.933579 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:50.933663 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:50.933772 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 00:10:50.933786 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 00:10:50.935920 9765 raft_consensus.cc:1261] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 1 FOLLOWER]: Refusing update from remote peer cb71f3c5b4984712b62a91d3c187ae5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:50.935972 10036 raft_consensus.cc:1261] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 1 FOLLOWER]: Refusing update from remote peer cb71f3c5b4984712b62a91d3c187ae5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:50.936322 10191 consensus_queue.cc:1035] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bddd8080d9eb43c69052726c1661f28a" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:50.936506 10195 consensus_queue.cc:1035] T 8144239d500d4a03b360d9c6c866a41e P cb71f3c5b4984712b62a91d3c187ae5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "505d89b0a244486497805e0cb7a89523" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 45071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:50.939669 10201 mvcc.cc:205] Tried to move back new op lower bound from 7317477175030509568 to 7317477174916939776. Current Snapshot: MvccSnapshot[applied={T|T < 7317477175030509568 or (T in {7317477175030509568})}]
I20260812 00:10:50.940104 10200 mvcc.cc:205] Tried to move back new op lower bound from 7317477175030509568 to 7317477174916939776. Current Snapshot: MvccSnapshot[applied={T|T < 7317477175030509568 or (T in {7317477175030509568})}]
I20260812 00:10:50.942072 9196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:50.942166 9196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:50.942984 9196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a19feb2173947dcbed1aece06ed195d: Sending DeleteTablet for 3 replicas of tablet 8144239d500d4a03b360d9c6c866a41e
I20260812 00:10:50.943356 9719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8144239d500d4a03b360d9c6c866a41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:50 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:53640
I20260812 00:10:50.943383 9444 tablet_service.cc:1514] Processing DeleteTablet for tablet 8144239d500d4a03b360d9c6c866a41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:50 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:47538
I20260812 00:10:50.943414 9998 tablet_service.cc:1514] Processing DeleteTablet for tablet 8144239d500d4a03b360d9c6c866a41e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:50 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:59266
I20260812 00:10:50.943640 10225 tablet_replica.cc:330] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: stopping tablet replica
I20260812 00:10:50.943748 10229 tablet_replica.cc:330] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: stopping tablet replica
I20260812 00:10:50.943780 10225 raft_consensus.cc:2229] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:50.943859 10229 raft_consensus.cc:2229] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:50.943922 10225 raft_consensus.cc:2258] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:50.943969 10229 raft_consensus.cc:2258] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:50.944110 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9320
I20260812 00:10:50.971287 10225 ts_tablet_manager.cc:1905] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:50.974911 10225 ts_tablet_manager.cc:1918] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:50.975061 10225 log.cc:1196] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/8144239d500d4a03b360d9c6c866a41e
I20260812 00:10:50.975353 10225 ts_tablet_manager.cc:1939] T 8144239d500d4a03b360d9c6c866a41e P 505d89b0a244486497805e0cb7a89523: Deleting consensus metadata
I20260812 00:10:50.975878 9169 catalog_manager.cc:4841] TS 505d89b0a244486497805e0cb7a89523 (127.7.85.194:45071): tablet 8144239d500d4a03b360d9c6c866a41e (table test-table [id=1e9d899a33dc492baa7b04517680e0a2]) successfully deleted
I20260812 00:10:50.980059 10229 ts_tablet_manager.cc:1905] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:50.982300 10229 ts_tablet_manager.cc:1918] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:50.982508 10229 log.cc:1196] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/8144239d500d4a03b360d9c6c866a41e
I20260812 00:10:50.982874 10229 ts_tablet_manager.cc:1939] T 8144239d500d4a03b360d9c6c866a41e P bddd8080d9eb43c69052726c1661f28a: Deleting consensus metadata
I20260812 00:10:50.985765 9170 catalog_manager.cc:4841] TS bddd8080d9eb43c69052726c1661f28a (127.7.85.195:44399): tablet 8144239d500d4a03b360d9c6c866a41e (table test-table [id=1e9d899a33dc492baa7b04517680e0a2]) successfully deleted
W20260812 00:10:50.997851 9171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:33167 (error 108)
I20260812 00:10:50.998064 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9603
I20260812 00:10:51.006347 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.006102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:33167 (local address 127.0.0.1:47554)
0812 00:10:51.006199 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.006279 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:33167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 00:10:51.006573 9171 catalog_manager.cc:4584] TS cb71f3c5b4984712b62a91d3c187ae5a (127.7.85.193:33167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8144239d500d4a03b360d9c6c866a41e: Network error: Client connection negotiation failed: client connection to 127.7.85.193:33167: connect: Connection refused (error 111)
I20260812 00:10:51.020453 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9872
I20260812 00:10:51.060402 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.060128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:33167 (local address 127.0.0.1:47560)
0812 00:10:51.060206 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.060249 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:33167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 00:10:51.062769 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.062624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:33167 (local address 127.0.0.1:47570)
0812 00:10:51.062688 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.062720 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:33167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 00:10:51.067301 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 9140
2026-08-12T00:10:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2546 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T00:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:51Z Disabled control of system clock
I20260812 00:10:51.123515 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33521
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:33521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:51.215060 10240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:51.215340 10240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:51.215365 10240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:51.216571 10240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:51.216617 10240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:51.216630 10240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:51.216643 10240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:51.216656 10240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:51.218103 10240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:33521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:33521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:51.218349 10240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:51.218616 10240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:51.224298 10253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:51.224370 10259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.224398 10240 server_base.cc:1034] running on GCE node
W20260812 00:10:51.224298 10252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.225289 10240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:51.225728 10240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:51.226948 10240 hybrid_clock.cc:648] HybridClock initialized: now 1786493451226918 us; error 42 us; skew 500 ppm
I20260812 00:10:51.228516 10240 webserver.cc:460] Webserver started at http://127.7.85.254:42089/ using document root <none> and password file <none>
I20260812 00:10:51.228794 10240 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:51.228889 10240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:51.229063 10240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:51.230268 10240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "625ee32e2be146faac47c6b8a30190ae"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "96ea8842a9cefe3cbb7ff774831946e6"
server_key_iv: "39b5c9c557dec15d917e9944afa4fc21"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.230659 10240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "625ee32e2be146faac47c6b8a30190ae"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "96ea8842a9cefe3cbb7ff774831946e6"
server_key_iv: "39b5c9c557dec15d917e9944afa4fc21"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.232116 10240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:10:51.232981 10272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.233259 10240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:51.233373 10240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "625ee32e2be146faac47c6b8a30190ae"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "96ea8842a9cefe3cbb7ff774831946e6"
server_key_iv: "39b5c9c557dec15d917e9944afa4fc21"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.233477 10240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:51.252321 10240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:51.252702 10240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:51.252871 10240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:51.257520 10240 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:33521
I20260812 00:10:51.257563 10372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:33521 every 8 connection(s)
I20260812 00:10:51.257948 10240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:51.259001 10373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:51.259682 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10240
I20260812 00:10:51.259835 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:51.260070 7511 external_mini_cluster.cc:1428] Setting key bcc0a26883e4d4169155dd5ea9336ccc
I20260812 00:10:51.261952 10373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae: Bootstrap starting.
I20260812 00:10:51.262765 10373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:51.263082 10373 log.cc:824] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:51.263868 10373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae: No bootstrap required, opened a new log
I20260812 00:10:51.266440 10373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625ee32e2be146faac47c6b8a30190ae" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33521 } }
I20260812 00:10:51.266702 10373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:51.266757 10373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625ee32e2be146faac47c6b8a30190ae, State: Initialized, Role: FOLLOWER
I20260812 00:10:51.266928 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.260895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36748 (local address 127.7.85.254:33521)
0812 00:10:51.261091 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.261098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.261124 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:10:51.261713 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.261721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.261787 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.261907 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:51.262259 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.263588 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.264445 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.265057 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.265240 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.266077 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:51.266083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:51.266084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:51.266313 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:51.266362 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:51.266377 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:51.266453 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.266593 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.266679 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260812 00:10:51.266907 10373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625ee32e2be146faac47c6b8a30190ae" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33521 } }
I20260812 00:10:51.266991 10373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:51.267024 10373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:51.267063 10373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:51.267761 10373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625ee32e2be146faac47c6b8a30190ae" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33521 } }
I20260812 00:10:51.267853 10373 leader_election.cc:302] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 625ee32e2be146faac47c6b8a30190ae; no voters:
I20260812 00:10:51.267987 10373 leader_election.cc:288] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:51.268062 10389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:51.268265 10389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [term 1 LEADER]: Becoming Leader. State: Replica: 625ee32e2be146faac47c6b8a30190ae, State: Running, Role: LEADER
I20260812 00:10:51.268317 10373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:51.268389 10389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625ee32e2be146faac47c6b8a30190ae" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33521 } }
I20260812 00:10:51.268769 10398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625ee32e2be146faac47c6b8a30190ae. Latest consensus state: current_term: 1 leader_uuid: "625ee32e2be146faac47c6b8a30190ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625ee32e2be146faac47c6b8a30190ae" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33521 } } }
I20260812 00:10:51.268810 10392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "625ee32e2be146faac47c6b8a30190ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625ee32e2be146faac47c6b8a30190ae" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 33521 } } }
I20260812 00:10:51.268965 10398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:51.268980 10392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:51.269264 10409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:51.270004 10409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:51.271729 10409 catalog_manager.cc:1347] Generated new cluster ID: 94a2b04ff2a5465691b629e9fcdb76e5
I20260812 00:10:51.271791 10409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:51.303321 10409 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:51.303918 10409 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:51.307384 10409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae: Generated new TSK 0
I20260812 00:10:51.307591 10409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:10:51.313030 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33521
--builtin_ntp_servers=127.7.85.212:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:51.409386 10426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:51.409618 10426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:51.409644 10426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:51.410890 10426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:51.410939 10426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:51.410966 10426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:51.412478 10426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:51.412786 10426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:51.413143 10426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:51.420292 10444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:51.420310 10438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.420580 10426 server_base.cc:1034] running on GCE node
W20260812 00:10:51.420346 10436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.420985 10426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:51.421352 10426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:51.422565 10426 hybrid_clock.cc:648] HybridClock initialized: now 1786493451422549 us; error 53 us; skew 500 ppm
I20260812 00:10:51.424221 10426 webserver.cc:460] Webserver started at http://127.7.85.193:33149/ using document root <none> and password file <none>
I20260812 00:10:51.424592 10426 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:51.424678 10426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:51.424849 10426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:51.426165 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "ea078783e1da47cfae99df483d51214b"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "83f272942a458d2ad2d752b01c7cd847"
server_key_iv: "dc4b5a163a9fe57b2f8ca6c9c31bbf76"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.426579 10426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "ea078783e1da47cfae99df483d51214b"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "83f272942a458d2ad2d752b01c7cd847"
server_key_iv: "dc4b5a163a9fe57b2f8ca6c9c31bbf76"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.428249 10426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:10:51.429129 10458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.429400 10426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:10:51.429495 10426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "ea078783e1da47cfae99df483d51214b"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "83f272942a458d2ad2d752b01c7cd847"
server_key_iv: "dc4b5a163a9fe57b2f8ca6c9c31bbf76"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.429550 10426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:51.441360 10426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:51.441684 10426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:51.441812 10426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:51.442015 10426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:51.442399 10426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:51.442436 10426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.442461 10426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:51.442499 10426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.449582 10426 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:45271
I20260812 00:10:51.449731 10690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:45271 every 8 connection(s)
I20260812 00:10:51.450137 10426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:51.455961 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.451004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:51891 (local address 127.7.85.254:33521)
0812 00:10:51.451057 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.451059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.451326 (+ 267us) server_negotiation.cc:407] Connection header received
0812 00:10:51.451877 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.451878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.451890 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.451957 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:51.452717 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.453342 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.454426 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.454627 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.454689 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.455614 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:51.455616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:51.455616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:51.455621 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:51.455637 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:51.455642 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:51.455701 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.455872 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.455897 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 00:10:51.456219 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.450943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33521 (local address 127.7.85.193:51891)
0812 00:10:51.451163 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.451211 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.451703 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.452041 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.452044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.452053 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:51.452600 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.452604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.453494 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.453496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.454314 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.454316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.454732 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.454733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.454853 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.455443 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:51.455443 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:51.455473 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:51.455496 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:51.455753 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:51.455779 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:51.455782 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.455856 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.455945 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260812 00:10:51.456913 10691 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33521
I20260812 00:10:51.457008 10691 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:51.457196 10691 heartbeater.cc:511] Master 127.7.85.254:33521 requested a full tablet report, sending...
I20260812 00:10:51.457880 10303 ts_manager.cc:194] Registered new tserver with Master: ea078783e1da47cfae99df483d51214b (127.7.85.193:45271)
I20260812 00:10:51.458544 10303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.193:51891
I20260812 00:10:51.460362 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10426
I20260812 00:10:51.460520 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:10:51.460754 7511 external_mini_cluster.cc:1428] Setting key a9d858be006fa700f8fd789a3656f26d
I20260812 00:10:51.462029 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33521
--builtin_ntp_servers=127.7.85.212:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:51.556005 10695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:51.556243 10695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:51.556267 10695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:51.557451 10695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:51.557494 10695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:51.557521 10695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:51.558866 10695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:51.559136 10695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:51.559432 10695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:51.565699 10708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.565817 10695 server_base.cc:1034] running on GCE node
W20260812 00:10:51.565675 10704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:51.565693 10705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.566461 10695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:51.566686 10695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:51.567862 10695 hybrid_clock.cc:648] HybridClock initialized: now 1786493451567848 us; error 34 us; skew 500 ppm
I20260812 00:10:51.569646 10695 webserver.cc:460] Webserver started at http://127.7.85.194:33785/ using document root <none> and password file <none>
I20260812 00:10:51.569954 10695 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:51.570006 10695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:51.570164 10695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:51.571285 10695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "309c615c969d4f96864f48ef9168ec12"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "7b0f4fc90154a720256b07d4379bbc01"
server_key_iv: "1cc81469de1ff7be398415dd4d692131"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.571653 10695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "309c615c969d4f96864f48ef9168ec12"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "7b0f4fc90154a720256b07d4379bbc01"
server_key_iv: "1cc81469de1ff7be398415dd4d692131"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.573084 10695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:10:51.574072 10722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.574334 10695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:51.574445 10695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "309c615c969d4f96864f48ef9168ec12"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "7b0f4fc90154a720256b07d4379bbc01"
server_key_iv: "1cc81469de1ff7be398415dd4d692131"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.574537 10695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:51.588322 10695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:51.588698 10695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:51.588877 10695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:51.589152 10695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:51.589578 10695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:51.589638 10695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.589677 10695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:51.589732 10695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.596297 10695 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35577
I20260812 00:10:51.596351 10931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35577 every 8 connection(s)
I20260812 00:10:51.596818 10695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:51.599792 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10695
I20260812 00:10:51.599927 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:10:51.600196 7511 external_mini_cluster.cc:1428] Setting key 512565e32b7e8d0a0f412dfe1db1962b
I20260812 00:10:51.601296 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33521
--builtin_ntp_servers=127.7.85.212:42059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:10:51.603377 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.597734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:60931 (local address 127.7.85.254:33521)
0812 00:10:51.597788 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.597790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.598001 (+ 211us) server_negotiation.cc:407] Connection header received
0812 00:10:51.598541 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.598543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.598555 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.598630 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:51.599439 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.600101 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.601629 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.601825 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.601894 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.602910 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:51.602912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:51.602912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:51.602917 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:51.602932 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:51.602936 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:51.602987 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.603086 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.603140 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:10:51.603497 10935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.597624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33521 (local address 127.7.85.194:60931)
0812 00:10:51.597835 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.597878 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.598416 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.598749 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.598753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.598762 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:51.599318 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.599323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.600229 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.600231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.601497 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.601501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.601931 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.601932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.602109 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.602744 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:51.602745 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:51.602786 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:51.602819 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:51.603006 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:51.603012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:51.603015 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.603088 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.603231 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260812 00:10:51.603943 10933 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33521
I20260812 00:10:51.604061 10933 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:51.604276 10933 heartbeater.cc:511] Master 127.7.85.254:33521 requested a full tablet report, sending...
I20260812 00:10:51.604701 10303 ts_manager.cc:194] Registered new tserver with Master: 309c615c969d4f96864f48ef9168ec12 (127.7.85.194:35577)
I20260812 00:10:51.605091 10303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.194:60931
W20260812 00:10:51.695212 10936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:51.695501 10936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:51.695544 10936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:51.696780 10936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:51.696854 10936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:51.696902 10936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:51.698385 10936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:33521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:51.698695 10936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:51.699141 10936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:51.705107 10945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:51.705103 10948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:51.705308 10946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:51.705579 10936 server_base.cc:1034] running on GCE node
I20260812 00:10:51.705945 10936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:51.706218 10936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:51.707391 10936 hybrid_clock.cc:648] HybridClock initialized: now 1786493451707351 us; error 56 us; skew 500 ppm
I20260812 00:10:51.709018 10936 webserver.cc:460] Webserver started at http://127.7.85.195:34287/ using document root <none> and password file <none>
I20260812 00:10:51.709331 10936 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:51.709388 10936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:51.709527 10936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:51.710667 10936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "d396a995f39c4d8dbede082558b956a4"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "e8ba4f325066726de0e53f4264adb9e9"
server_key_iv: "ffef5d1bd06d3f71789360d8ac8abd7c"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.711042 10936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "d396a995f39c4d8dbede082558b956a4"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "e8ba4f325066726de0e53f4264adb9e9"
server_key_iv: "ffef5d1bd06d3f71789360d8ac8abd7c"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.712493 10936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:10:51.713503 10962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.713737 10936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:51.713866 10936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "d396a995f39c4d8dbede082558b956a4"
format_stamp: "Formatted at 2026-08-12 00:10:51 on dist-test-slave-z8x9"
server_key: "e8ba4f325066726de0e53f4264adb9e9"
server_key_iv: "ffef5d1bd06d3f71789360d8ac8abd7c"
server_key_version: "encryptionkey@0"
I20260812 00:10:51.713956 10936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:51.727739 10936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:51.728137 10936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:51.728335 10936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:51.728643 10936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:51.729025 10936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:51.729089 10936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.729136 10936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:51.729199 10936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:51.735869 10936 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:38895
I20260812 00:10:51.735970 11186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:38895 every 8 connection(s)
I20260812 00:10:51.736321 10936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:51.738632 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 10936
I20260812 00:10:51.738730 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:10:51.738948 7511 external_mini_cluster.cc:1428] Setting key c29065187a4c5847cacf15684e8793c3
I20260812 00:10:51.742652 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.737282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:44125 (local address 127.7.85.254:33521)
0812 00:10:51.737359 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.737362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.737783 (+ 421us) server_negotiation.cc:407] Connection header received
0812 00:10:51.738187 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.738188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.738211 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.738272 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:51.739165 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.739903 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.740940 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.741157 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.741267 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.742325 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:51.742327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:51.742327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:51.742333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:51.742349 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:51.742354 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:51.742409 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.742538 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.742566 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 00:10:51.742910 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.737251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:33521 (local address 127.7.85.195:44125)
0812 00:10:51.737500 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.737537 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.738059 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.738365 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.738368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.738378 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:51.739041 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.739049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.740000 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.740000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.740822 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.740825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.741294 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.741295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.741455 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.742087 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:51.742087 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:51.742121 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:51.742147 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:51.742418 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:51.742425 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:51.742429 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.742514 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.742634 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260812 00:10:51.743649 11188 heartbeater.cc:347] Connected to a master server at 127.7.85.254:33521
I20260812 00:10:51.743768 11188 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:51.743937 11188 heartbeater.cc:511] Master 127.7.85.254:33521 requested a full tablet report, sending...
I20260812 00:10:51.744379 10303 ts_manager.cc:194] Registered new tserver with Master: d396a995f39c4d8dbede082558b956a4 (127.7.85.195:38895)
I20260812 00:10:51.744831 10303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.195:44125
I20260812 00:10:51.750941 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:51.755584 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.752488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.7.85.254:33521)
0812 00:10:51.752541 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.752543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.752555 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:10:51.752635 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.752636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.752647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.752715 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:51.752900 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.753464 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.754235 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.754423 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.754500 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.755288 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:51.755289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:51.755290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:51.755295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:51.755310 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:51.755315 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:51.755410 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.755477 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.755488 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 00:10:51.757583 10303 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:51.759291 10303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:51.766073 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.761666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.7.85.194:35577)
0812 00:10:51.761863 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.761868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.762025 (+ 157us) server_negotiation.cc:407] Connection header received
0812 00:10:51.762171 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.762174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.762191 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.762274 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:51.762463 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.763748 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765004 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.765625 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765743 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.765844 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.765943 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.765981 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260812 00:10:51.766063 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.761584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35577 (local address 127.0.0.1:52676)
0812 00:10:51.761947 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.761959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.762056 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.762287 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.762290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.762293 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:51.762369 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.762371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.763834 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.763835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.764908 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.764911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.765746 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.765830 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.765845 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.765923 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.765979 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260812 00:10:51.766340 11207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.761580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:38895 (local address 127.0.0.1:38118)
0812 00:10:51.761865 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.761909 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.761980 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.762380 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.762381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.762383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:51.762433 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.762434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.763493 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.763494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.764912 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.764914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.766004 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.766005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.766072 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.766078 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.766133 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.766266 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":24,"thread_start_us":72,"threads_started":1}
I20260812 00:10:51.766449 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.761810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:45271 (local address 127.0.0.1:44158)
0812 00:10:51.762120 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.762129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.762168 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.762761 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.762763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.762765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:51.762802 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.762803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.764009 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.764011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.765076 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.765079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.766219 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.766220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.766287 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.766291 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.766348 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.766387 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":84,"threads_started":1}
I20260812 00:10:51.766705 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.762045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.7.85.193:45271)
0812 00:10:51.762214 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.762218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.762227 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:10:51.762471 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.762474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.762493 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.762743 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:51.762872 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.763924 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765189 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.765891 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765960 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.766040 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.766548 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.766593 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260812 00:10:51.766940 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.761695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.7.85.195:38895)
0812 00:10:51.761944 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.761947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.761960 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:10:51.762281 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.762284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.762298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.762368 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:51.762513 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.763397 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765014 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.765797 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.765870 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.765964 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.766622 (+ 658us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.766671 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":92,"threads_started":1}
I20260812 00:10:51.766960 10814 tablet_service.cc:1467] Processing CreateTablet for tablet b7acec16d1ba40008e07868f2ec8e2e9 (DEFAULT_TABLE table=test-table [id=274b6d10c20f470e9f6242d2dcfe0a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:51.767408 10814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7acec16d1ba40008e07868f2ec8e2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:51.767567 11060 tablet_service.cc:1467] Processing CreateTablet for tablet b7acec16d1ba40008e07868f2ec8e2e9 (DEFAULT_TABLE table=test-table [id=274b6d10c20f470e9f6242d2dcfe0a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:51.767884 11060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7acec16d1ba40008e07868f2ec8e2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:51.770078 10566 tablet_service.cc:1467] Processing CreateTablet for tablet b7acec16d1ba40008e07868f2ec8e2e9 (DEFAULT_TABLE table=test-table [id=274b6d10c20f470e9f6242d2dcfe0a4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:51.770459 10566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7acec16d1ba40008e07868f2ec8e2e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:51.770802 11217 tablet_bootstrap.cc:492] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Bootstrap starting.
I20260812 00:10:51.771142 11218 tablet_bootstrap.cc:492] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Bootstrap starting.
I20260812 00:10:51.771564 11217 tablet_bootstrap.cc:654] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:51.771795 11218 tablet_bootstrap.cc:654] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:51.771911 11217 log.cc:824] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:51.772105 11218 log.cc:824] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:51.772742 11217 tablet_bootstrap.cc:492] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: No bootstrap required, opened a new log
I20260812 00:10:51.772791 11217 ts_tablet_manager.cc:1397] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:10:51.772877 11218 tablet_bootstrap.cc:492] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: No bootstrap required, opened a new log
I20260812 00:10:51.772938 11218 ts_tablet_manager.cc:1397] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:10:51.773874 11225 tablet_bootstrap.cc:492] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: Bootstrap starting.
I20260812 00:10:51.774569 11225 tablet_bootstrap.cc:654] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:51.774948 11225 log.cc:824] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:51.775300 11217 raft_consensus.cc:348] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.775441 11217 raft_consensus.cc:374] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:51.775509 11217 raft_consensus.cc:729] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309c615c969d4f96864f48ef9168ec12, State: Initialized, Role: FOLLOWER
I20260812 00:10:51.775707 11217 consensus_queue.cc:260] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.775807 11218 raft_consensus.cc:348] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.776010 11217 ts_tablet_manager.cc:1428] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:51.776031 11218 raft_consensus.cc:374] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:51.776052 11225 tablet_bootstrap.cc:492] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: No bootstrap required, opened a new log
I20260812 00:10:51.776075 11218 raft_consensus.cc:729] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d396a995f39c4d8dbede082558b956a4, State: Initialized, Role: FOLLOWER
I20260812 00:10:51.776113 11225 ts_tablet_manager.cc:1397] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:10:51.776142 10933 heartbeater.cc:503] Master 127.7.85.254:33521 was elected leader, sending a full tablet report...
I20260812 00:10:51.776196 11218 consensus_queue.cc:260] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.776634 11218 ts_tablet_manager.cc:1428] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 00:10:51.776670 11188 heartbeater.cc:503] Master 127.7.85.254:33521 was elected leader, sending a full tablet report...
I20260812 00:10:51.778725 11225 raft_consensus.cc:348] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.778862 11225 raft_consensus.cc:374] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:51.778884 11225 raft_consensus.cc:729] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea078783e1da47cfae99df483d51214b, State: Initialized, Role: FOLLOWER
I20260812 00:10:51.779001 11225 consensus_queue.cc:260] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.779268 11225 ts_tablet_manager.cc:1428] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:51.779354 10691 heartbeater.cc:503] Master 127.7.85.254:33521 was elected leader, sending a full tablet report...
I20260812 00:10:51.796499 11234 raft_consensus.cc:482] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:51.796657 11234 raft_consensus.cc:504] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.797106 11234 leader_election.cc:288] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 309c615c969d4f96864f48ef9168ec12 (127.7.85.194:35577), d396a995f39c4d8dbede082558b956a4 (127.7.85.195:38895)
I20260812 00:10:51.800410 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.797380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35577 (local address 127.7.85.193:40621)
0812 00:10:51.797446 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.797460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.797507 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.797708 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.797709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.797712 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:51.797801 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.797803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.798593 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.798595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.799484 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.799487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.800128 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.800129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.800212 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.800217 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.800274 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.800317 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":63}
I20260812 00:10:51.800446 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.797460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:40621 (local address 127.7.85.194:35577)
0812 00:10:51.797514 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.797516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.797560 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:10:51.797592 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.797593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.797606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.797667 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:51.797910 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.798488 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.799590 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.800022 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.800084 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.800091 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.800290 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.800370 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 00:10:51.800558 11235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.797381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:38895 (local address 127.7.85.193:49225)
0812 00:10:51.797610 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.797622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:51.797711 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:51.797910 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:51.797911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:51.797913 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:51.797984 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.797985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.798787 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.798789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.799622 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:51.799624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.800237 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.800238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:51.800335 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.800340 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:10:51.800400 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 00:10:51.800437 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":23,"thread_start_us":129,"threads_started":1}
I20260812 00:10:51.800565 11212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.797460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:49225 (local address 127.7.85.195:38895)
0812 00:10:51.797526 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.797528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.797727 (+ 199us) server_negotiation.cc:407] Connection header received
0812 00:10:51.797781 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.797782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.797792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.797852 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:51.798144 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.798681 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.799728 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.800143 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.800225 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.800233 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.800429 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.800482 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:10:51.800830 10853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7acec16d1ba40008e07868f2ec8e2e9" candidate_uuid: "ea078783e1da47cfae99df483d51214b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "309c615c969d4f96864f48ef9168ec12" is_pre_election: true
I20260812 00:10:51.800886 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7acec16d1ba40008e07868f2ec8e2e9" candidate_uuid: "ea078783e1da47cfae99df483d51214b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d396a995f39c4d8dbede082558b956a4" is_pre_election: true
I20260812 00:10:51.801033 11099 raft_consensus.cc:2454] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea078783e1da47cfae99df483d51214b in term 0.
I20260812 00:10:51.801033 10853 raft_consensus.cc:2454] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea078783e1da47cfae99df483d51214b in term 0.
I20260812 00:10:51.801312 10471 leader_election.cc:302] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309c615c969d4f96864f48ef9168ec12, ea078783e1da47cfae99df483d51214b; no voters:
I20260812 00:10:51.801460 11234 raft_consensus.cc:2781] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:51.801522 11234 raft_consensus.cc:482] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:51.801546 11234 raft_consensus.cc:3037] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:51.802304 11234 raft_consensus.cc:504] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.802492 11234 leader_election.cc:288] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [CANDIDATE]: Term 1 election: Requested vote from peers 309c615c969d4f96864f48ef9168ec12 (127.7.85.194:35577), d396a995f39c4d8dbede082558b956a4 (127.7.85.195:38895)
I20260812 00:10:51.802696 10853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7acec16d1ba40008e07868f2ec8e2e9" candidate_uuid: "ea078783e1da47cfae99df483d51214b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "309c615c969d4f96864f48ef9168ec12"
I20260812 00:10:51.802742 11099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7acec16d1ba40008e07868f2ec8e2e9" candidate_uuid: "ea078783e1da47cfae99df483d51214b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d396a995f39c4d8dbede082558b956a4"
I20260812 00:10:51.802786 10853 raft_consensus.cc:3037] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:51.802811 11099 raft_consensus.cc:3037] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:51.803733 11099 raft_consensus.cc:2454] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea078783e1da47cfae99df483d51214b in term 1.
I20260812 00:10:51.803733 10853 raft_consensus.cc:2454] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea078783e1da47cfae99df483d51214b in term 1.
I20260812 00:10:51.803988 10471 leader_election.cc:302] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 309c615c969d4f96864f48ef9168ec12, ea078783e1da47cfae99df483d51214b; no voters:
I20260812 00:10:51.804143 11234 raft_consensus.cc:2781] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:51.804363 11234 raft_consensus.cc:686] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 1 LEADER]: Becoming Leader. State: Replica: ea078783e1da47cfae99df483d51214b, State: Running, Role: LEADER
I20260812 00:10:51.804491 11234 consensus_queue.cc:237] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } }
I20260812 00:10:51.805564 10297 catalog_manager.cc:5495] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b reported cstate change: term changed from 0 to 1, leader changed from <none> to ea078783e1da47cfae99df483d51214b (127.7.85.193). New cstate: current_term: 1 leader_uuid: "ea078783e1da47cfae99df483d51214b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea078783e1da47cfae99df483d51214b" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 45271 } health_report { overall_health: HEALTHY } } }
I20260812 00:10:51.818085 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.815245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.7.85.193:45271)
0812 00:10:51.815300 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:51.815303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:51.815339 (+ 36us) server_negotiation.cc:407] Connection header received
0812 00:10:51.815415 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:51.815416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:51.815436 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:51.815510 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:51.815692 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.816221 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.817089 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:51.817414 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:51.817479 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:51.817643 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:51.817737 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:51.817817 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:51.817874 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 00:10:51.817884 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:10:51.820504 10853 raft_consensus.cc:1261] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 1 FOLLOWER]: Refusing update from remote peer ea078783e1da47cfae99df483d51214b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:51.820516 11099 raft_consensus.cc:1261] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 1 FOLLOWER]: Refusing update from remote peer ea078783e1da47cfae99df483d51214b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:51.820955 11236 consensus_queue.cc:1035] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [LEADER]: Connected to new peer: Peer: permanent_uuid: "309c615c969d4f96864f48ef9168ec12" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:51.821187 11234 consensus_queue.cc:1035] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d396a995f39c4d8dbede082558b956a4" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 38895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:51.824270 11242 mvcc.cc:205] Tried to move back new op lower bound from 7317477178652487680 to 7317477178592059392. Current Snapshot: MvccSnapshot[applied={T|T < 7317477178652487680}]
I20260812 00:10:51.826745 10303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:51.826839 10303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:51.827726 10303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 625ee32e2be146faac47c6b8a30190ae: Sending DeleteTablet for 3 replicas of tablet b7acec16d1ba40008e07868f2ec8e2e9
I20260812 00:10:51.828114 11060 tablet_service.cc:1514] Processing DeleteTablet for tablet b7acec16d1ba40008e07868f2ec8e2e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:51 UTC) from {username='slave'} at 127.0.0.1:38118
I20260812 00:10:51.828207 10814 tablet_service.cc:1514] Processing DeleteTablet for tablet b7acec16d1ba40008e07868f2ec8e2e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:51 UTC) from {username='slave'} at 127.0.0.1:52676
I20260812 00:10:51.828265 10566 tablet_service.cc:1514] Processing DeleteTablet for tablet b7acec16d1ba40008e07868f2ec8e2e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:51 UTC) from {username='slave'} at 127.0.0.1:44158
I20260812 00:10:51.828493 11265 tablet_replica.cc:330] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b: stopping tablet replica
I20260812 00:10:51.828611 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10426
I20260812 00:10:51.828646 11265 raft_consensus.cc:2229] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:10:51.828764 11265 raft_consensus.cc:2258] T b7acec16d1ba40008e07868f2ec8e2e9 P ea078783e1da47cfae99df483d51214b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:51.829294 11260 tablet_replica.cc:330] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: stopping tablet replica
I20260812 00:10:51.829393 11260 raft_consensus.cc:2229] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:51.829509 11260 raft_consensus.cc:2258] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:51.829700 11264 tablet_replica.cc:330] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: stopping tablet replica
I20260812 00:10:51.829806 11264 raft_consensus.cc:2229] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:51.829895 11264 raft_consensus.cc:2258] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:51.832623 11260 ts_tablet_manager.cc:1905] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:51.834817 11264 ts_tablet_manager.cc:1905] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:51.837342 11264 ts_tablet_manager.cc:1918] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:51.837445 11264 log.cc:1196] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/b7acec16d1ba40008e07868f2ec8e2e9
I20260812 00:10:51.837692 11264 ts_tablet_manager.cc:1939] T b7acec16d1ba40008e07868f2ec8e2e9 P 309c615c969d4f96864f48ef9168ec12: Deleting consensus metadata
I20260812 00:10:51.838757 10278 catalog_manager.cc:4841] TS 309c615c969d4f96864f48ef9168ec12 (127.7.85.194:35577): tablet b7acec16d1ba40008e07868f2ec8e2e9 (table test-table [id=274b6d10c20f470e9f6242d2dcfe0a4d]) successfully deleted
I20260812 00:10:51.850073 11260 ts_tablet_manager.cc:1918] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:51.850230 11260 log.cc:1196] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/b7acec16d1ba40008e07868f2ec8e2e9
I20260812 00:10:51.850524 11260 ts_tablet_manager.cc:1939] T b7acec16d1ba40008e07868f2ec8e2e9 P d396a995f39c4d8dbede082558b956a4: Deleting consensus metadata
I20260812 00:10:51.851117 10276 catalog_manager.cc:4841] TS d396a995f39c4d8dbede082558b956a4 (127.7.85.195:38895): tablet b7acec16d1ba40008e07868f2ec8e2e9 (table test-table [id=274b6d10c20f470e9f6242d2dcfe0a4d]) successfully deleted
W20260812 00:10:51.854879 10278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:45271 (error 108)
I20260812 00:10:51.855463 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.855324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:45271 (local address 127.0.0.1:44170)
0812 00:10:51.855388 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.855431 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:45271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:10:51.855564 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10695
W20260812 00:10:51.855604 10278 catalog_manager.cc:4584] TS ea078783e1da47cfae99df483d51214b (127.7.85.193:45271): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7acec16d1ba40008e07868f2ec8e2e9: Network error: Client connection negotiation failed: client connection to 127.7.85.193:45271: connect: Connection refused (error 111)
I20260812 00:10:51.886809 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10936
I20260812 00:10:51.908174 11209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:51.907998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:45271 (local address 127.0.0.1:44180)
0812 00:10:51.908097 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:51.908129 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:45271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:10:51.908175 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 10240
2026-08-12T00:10:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (833 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:10:51 dist-test-slave-z8x9 krb5kdc[11271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:10:51 dist-test-slave-z8x9 krb5kdc[11271](info): set up 2 sockets
Aug 12 00:10:51 dist-test-slave-z8x9 krb5kdc[11271](info): commencing operation
krb5kdc: starting...
W20260812 00:10:53.279234 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:10:53 dist-test-slave-z8x9 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:10:53.457819 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:45913
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:53.546005 11300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:53.546309 11300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:53.546355 11300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:53.547638 11300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:53.547721 11300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:53.547749 11300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:53.547782 11300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:53.547808 11300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:53.549433 11300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:53.549736 11300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:53.549989 11300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:53.555457 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:53.555464 11313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:53.555512 11309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:53.556250 11300 server_base.cc:1034] running on GCE node
I20260812 00:10:53.556708 11300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:53.557062 11300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:53.558291 11300 hybrid_clock.cc:648] HybridClock initialized: now 1786493453558268 us; error 43 us; skew 500 ppm
Aug 12 00:10:53 dist-test-slave-z8x9 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:53.561616 11300 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:10:53.562186 11300 webserver.cc:460] Webserver started at http://127.7.85.254:41171/ using document root <none> and password file <none>
I20260812 00:10:53.563367 11300 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:53.563472 11300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:53.564124 11300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:53.565989 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "abf610ebc4574cdca56a499984cc5ec1"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "5c278ce3d226f1fb28e9fc98d66401fa"
server_key_iv: "49c331817a6a16e4906ddd21867e77ff"
server_key_version: "encryptionkey@0"
I20260812 00:10:53.566823 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "abf610ebc4574cdca56a499984cc5ec1"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "5c278ce3d226f1fb28e9fc98d66401fa"
server_key_iv: "49c331817a6a16e4906ddd21867e77ff"
server_key_version: "encryptionkey@0"
I20260812 00:10:53.568488 11300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:10:53.569572 11326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:53.569849 11300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:53.569976 11300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "abf610ebc4574cdca56a499984cc5ec1"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "5c278ce3d226f1fb28e9fc98d66401fa"
server_key_iv: "49c331817a6a16e4906ddd21867e77ff"
server_key_version: "encryptionkey@0"
I20260812 00:10:53.570061 11300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:53.585652 11300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:53.588747 11300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:53.588965 11300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:53.594831 11300 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:45913
I20260812 00:10:53.594920 11442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:45913 every 8 connection(s)
I20260812 00:10:53.595364 11300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:53.596427 11445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:53.600675 11445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1: Bootstrap starting.
I20260812 00:10:53.601403 11445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:53.601663 11445 log.cc:824] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:53.603083 11445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1: No bootstrap required, opened a new log
I20260812 00:10:53.605566 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11300
I20260812 00:10:53.605697 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:53.605082 11445 raft_consensus.cc:348] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf610ebc4574cdca56a499984cc5ec1" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45913 } }
I20260812 00:10:53.605950 7511 external_mini_cluster.cc:1428] Setting key 760da6c9f80cdbd102c3d6b2fc4e2bd0
I20260812 00:10:53.605873 11445 raft_consensus.cc:374] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:53.606000 11445 raft_consensus.cc:729] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abf610ebc4574cdca56a499984cc5ec1, State: Initialized, Role: FOLLOWER
I20260812 00:10:53.606254 11445 consensus_queue.cc:260] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf610ebc4574cdca56a499984cc5ec1" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45913 } }
I20260812 00:10:53.606395 11445 raft_consensus.cc:388] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:53.606444 11445 raft_consensus.cc:482] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:53.606503 11445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:53.607447 11445 raft_consensus.cc:504] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf610ebc4574cdca56a499984cc5ec1" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45913 } }
I20260812 00:10:53.607738 11445 leader_election.cc:302] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abf610ebc4574cdca56a499984cc5ec1; no voters:
I20260812 00:10:53.608009 11445 leader_election.cc:288] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:53.608035 11453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:53.608168 11453 raft_consensus.cc:686] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [term 1 LEADER]: Becoming Leader. State: Replica: abf610ebc4574cdca56a499984cc5ec1, State: Running, Role: LEADER
I20260812 00:10:53.608350 11445 sys_catalog.cc:564] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:53.608353 11453 consensus_queue.cc:237] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf610ebc4574cdca56a499984cc5ec1" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45913 } }
I20260812 00:10:53.608913 11455 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abf610ebc4574cdca56a499984cc5ec1. Latest consensus state: current_term: 1 leader_uuid: "abf610ebc4574cdca56a499984cc5ec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf610ebc4574cdca56a499984cc5ec1" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45913 } } }
I20260812 00:10:53.608927 11462 sys_catalog.cc:455] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abf610ebc4574cdca56a499984cc5ec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abf610ebc4574cdca56a499984cc5ec1" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45913 } } }
I20260812 00:10:53.609148 11462 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:53.609131 11455 sys_catalog.cc:458] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:53.610095 11473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:53.610683 11473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:53.612601 11473 catalog_manager.cc:1347] Generated new cluster ID: b3b38d3ce6b4476d8e32d72e83613008
I20260812 00:10:53.612660 11473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:10:53 dist-test-slave-z8x9 krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:53.622152 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:53.606621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.7.85.254:45913)
0812 00:10:53.606838 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:53.606850 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:53.606933 (+ 83us) server_negotiation.cc:407] Connection header received
0812 00:10:53.607383 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:53.607391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:53.607445 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:53.607608 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:53.608460 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:53.609822 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:53.611173 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:53.611716 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:53.611979 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:53.616004 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:53.616011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:53.616011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:53.616030 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:53.619768 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:53.620599 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:53.620604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:53.620604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:53.620774 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:53.621007 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:53.621008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:53.621009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:53.621433 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:53.621618 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:53.621709 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 00:10:53.621882 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":113,"threads_started":1}
I20260812 00:10:53.634826 11473 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:53.635651 11473 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:53.641968 11473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1: Generated new TSK 0
I20260812 00:10:53.642230 11473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.85.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
Entry for principal kudu/127.7.85.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab.
I20260812 00:10:53.686880 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45913
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:53.792933 11494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:53.793252 11494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:53.793313 11494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:53.794689 11494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:53.794770 11494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:53.794816 11494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:53.796492 11494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.193.keytab
--principal=kudu/127.7.85.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:53.796823 11494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:53.797097 11494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:53.803489 11500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:53.803567 11494 server_base.cc:1034] running on GCE node
W20260812 00:10:53.803519 11503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:53.803519 11501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:53.804343 11494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:53.804679 11494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:53.805917 11494 hybrid_clock.cc:648] HybridClock initialized: now 1786493453805912 us; error 45 us; skew 500 ppm
Aug 12 00:10:53 dist-test-slave-z8x9 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:53.809821 11494 init.cc:377] Logged in from keytab as kudu/127.7.85.193@KRBTEST.COM (short username kudu)
I20260812 00:10:53.810406 11494 webserver.cc:460] Webserver started at http://127.7.85.193:34921/ using document root <none> and password file <none>
I20260812 00:10:53.810698 11494 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:53.810773 11494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:53.810940 11494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:53.812062 11494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "8f281b408a5544d1a5206cbcc44d8e2d"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "3a3d10ce5ccd7cd918460dfdf3b8b2a9"
server_key_iv: "5725488aa4f019ce690719a7d04ca47e"
server_key_version: "encryptionkey@0"
I20260812 00:10:53.812474 11494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "8f281b408a5544d1a5206cbcc44d8e2d"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "3a3d10ce5ccd7cd918460dfdf3b8b2a9"
server_key_iv: "5725488aa4f019ce690719a7d04ca47e"
server_key_version: "encryptionkey@0"
I20260812 00:10:53.814292 11494 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:10:53.815564 11514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:53.815887 11494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:53.816031 11494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "8f281b408a5544d1a5206cbcc44d8e2d"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "3a3d10ce5ccd7cd918460dfdf3b8b2a9"
server_key_iv: "5725488aa4f019ce690719a7d04ca47e"
server_key_version: "encryptionkey@0"
I20260812 00:10:53.816126 11494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:53.828562 11494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:53.832242 11494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:53.832463 11494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:53.832762 11494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:53.833127 11494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:53.833217 11494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:53.833292 11494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:53.833324 11494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:53.842288 11494 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:37141
I20260812 00:10:53.842456 11714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:37141 every 8 connection(s)
I20260812 00:10:53.842818 11494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:53.845093 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11494
I20260812 00:10:53.845259 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:10:53.845508 7511 external_mini_cluster.cc:1428] Setting key 10173ae476e756f3326c27d7d9929883
Aug 12 00:10:53 dist-test-slave-z8x9 krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.193@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:53.856103 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:53.843819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:39687 (local address 127.7.85.254:45913)
0812 00:10:53.843865 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:53.843867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:53.844086 (+ 219us) server_negotiation.cc:407] Connection header received
0812 00:10:53.845344 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:53.845347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:53.845366 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:53.845705 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:53.846593 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:53.847138 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:53.848878 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:53.849276 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:53.849365 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:53.852365 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:53.852368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:53.852368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:53.852379 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:53.854347 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:53.854844 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:53.854846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:53.854846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:53.854920 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:53.855238 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:53.855239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:53.855239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:53.855422 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:53.855503 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:53.855867 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 00:10:53.855972 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:10:53.856197 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:53.843776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:45913 (local address 127.7.85.193:39687)
0812 00:10:53.843951 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:53.843992 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:53.845215 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:53.845793 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:53.845798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:53.846148 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:53.846445 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:53.846451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:53.847253 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:53.847255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:53.848755 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:53.848760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:53.849448 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:53.849449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:53.849570 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:53.850355 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:53.850365 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:53.852243 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:53.854458 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:53.854467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:53.854469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:53.854742 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:53.855017 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:53.855018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:53.855019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:53.855152 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:53.855524 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:53.855531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:53.855688 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 00:10:53.855816 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 00:10:53.855974 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260812 00:10:53.856848 11715 heartbeater.cc:347] Connected to a master server at 127.7.85.254:45913
I20260812 00:10:53.856974 11715 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.85.194@KRBTEST.COM; defaulting to no policy
I20260812 00:10:53.857260 11715 heartbeater.cc:511] Master 127.7.85.254:45913 requested a full tablet report, sending...
I20260812 00:10:53.858417 11371 ts_manager.cc:194] Registered new tserver with Master: 8f281b408a5544d1a5206cbcc44d8e2d (127.7.85.193:37141)
I20260812 00:10:53.859453 11371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.193@KRBTEST.COM'} at 127.7.85.193:39687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
Entry for principal kudu/127.7.85.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab.
I20260812 00:10:53.880752 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45913
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:53.978359 11725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:53.978606 11725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:53.978634 11725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:53.981125 11725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:53.981293 11725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:53.981338 11725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:53.984928 11725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.194.keytab
--principal=kudu/127.7.85.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:53.985550 11725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:53.985900 11725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:53.992497 11746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:53.992630 11725 server_base.cc:1034] running on GCE node
W20260812 00:10:53.992489 11741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:53.992489 11736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:53.993028 11725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:53.993372 11725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:53.994625 11725 hybrid_clock.cc:648] HybridClock initialized: now 1786493453994579 us; error 67 us; skew 500 ppm
Aug 12 00:10:53 dist-test-slave-z8x9 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:53.998015 11725 init.cc:377] Logged in from keytab as kudu/127.7.85.194@KRBTEST.COM (short username kudu)
I20260812 00:10:53.998557 11725 webserver.cc:460] Webserver started at http://127.7.85.194:36205/ using document root <none> and password file <none>
I20260812 00:10:53.998790 11725 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:53.998836 11725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:53.999073 11725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:54.000353 11725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "d8553ef067904111879ffa1bbc5de705"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "c2c1d80ce64f109323be984e5aa21c57"
server_key_iv: "7d2c1ec924c57dcce8e295cc36bfffe5"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.000676 11725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "d8553ef067904111879ffa1bbc5de705"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "c2c1d80ce64f109323be984e5aa21c57"
server_key_iv: "7d2c1ec924c57dcce8e295cc36bfffe5"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.002208 11725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260812 00:10:54.003155 11763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.003392 11725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:54.003496 11725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "d8553ef067904111879ffa1bbc5de705"
format_stamp: "Formatted at 2026-08-12 00:10:53 on dist-test-slave-z8x9"
server_key: "c2c1d80ce64f109323be984e5aa21c57"
server_key_iv: "7d2c1ec924c57dcce8e295cc36bfffe5"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.003564 11725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:54.044485 11725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:54.047242 11725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:54.047381 11725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:54.047591 11725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:54.047951 11725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:54.047991 11725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.048017 11725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:54.048070 11725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.054684 11725 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35197
I20260812 00:10:54.054776 11976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35197 every 8 connection(s)
I20260812 00:10:54.055131 11725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:54.061264 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11725
I20260812 00:10:54.061453 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:10:54.061932 7511 external_mini_cluster.cc:1428] Setting key e8ebf226cc653ab90994b2647088367d
Aug 12 00:10:54 dist-test-slave-z8x9 krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.194@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:54.067847 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.056115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:51209 (local address 127.7.85.254:45913)
0812 00:10:54.056201 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.056203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.056399 (+ 196us) server_negotiation.cc:407] Connection header received
0812 00:10:54.056897 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.056898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.056910 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.056980 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:54.058186 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.058776 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.059962 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.060186 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.060265 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.063874 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:54.063876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:54.063876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:54.063887 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:54.065866 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:54.066550 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:54.066551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:54.066551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:54.066614 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:54.066930 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:54.066930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:54.066931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:54.067055 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:54.067110 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.067610 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.067754 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 00:10:54.068068 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.055927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:45913 (local address 127.7.85.194:51209)
0812 00:10:54.056212 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.056265 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.056766 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.057068 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.057071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.057811 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:54.058029 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.058034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.058932 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.058934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.059829 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.059832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.060350 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.060352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.060553 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.061652 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:54.061666 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:54.063729 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:54.065985 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:54.066001 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:54.066005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:54.066453 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:54.066690 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:54.066691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:54.066691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:54.066858 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:54.067125 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:54.067133 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:54.067335 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.067467 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.067699 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":118,"threads_started":1}
I20260812 00:10:54.068943 11977 heartbeater.cc:347] Connected to a master server at 127.7.85.254:45913
I20260812 00:10:54.069082 11977 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:54.069344 11977 heartbeater.cc:511] Master 127.7.85.254:45913 requested a full tablet report, sending...
I20260812 00:10:54.069855 11371 ts_manager.cc:194] Registered new tserver with Master: d8553ef067904111879ffa1bbc5de705 (127.7.85.194:35197)
I20260812 00:10:54.070492 11371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.194@KRBTEST.COM'} at 127.7.85.194:51209
WARNING: no policy specified for kudu/127.7.85.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
Entry for principal kudu/127.7.85.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab.
I20260812 00:10:54.098641 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45913
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:54.195407 11992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:54.195638 11992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:54.195663 11992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:54.196867 11992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:54.196913 11992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:54.196941 11992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:54.198552 11992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.195.keytab
--principal=kudu/127.7.85.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:54.198824 11992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:54.199162 11992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:54.204814 12011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:54.204991 12005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:54.204988 11992 server_base.cc:1034] running on GCE node
W20260812 00:10:54.204814 12007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:54.205436 11992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:54.205711 11992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:54.206911 11992 hybrid_clock.cc:648] HybridClock initialized: now 1786493454206905 us; error 70 us; skew 500 ppm
Aug 12 00:10:54 dist-test-slave-z8x9 krb5kdc[11271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:54.210142 11992 init.cc:377] Logged in from keytab as kudu/127.7.85.195@KRBTEST.COM (short username kudu)
I20260812 00:10:54.210587 11992 webserver.cc:460] Webserver started at http://127.7.85.195:41019/ using document root <none> and password file <none>
I20260812 00:10:54.210790 11992 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:54.210834 11992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:54.210927 11992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:54.212025 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "0b1d4e6415834c068c74ee91d9fef793"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "e822b441392af274dd1e1b709620e33f"
server_key_iv: "bcf136aa12548385b12475355bd5726a"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.212391 11992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "0b1d4e6415834c068c74ee91d9fef793"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "e822b441392af274dd1e1b709620e33f"
server_key_iv: "bcf136aa12548385b12475355bd5726a"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.213902 11992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:10:54.214972 12026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.215195 11992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:54.215307 11992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "0b1d4e6415834c068c74ee91d9fef793"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "e822b441392af274dd1e1b709620e33f"
server_key_iv: "bcf136aa12548385b12475355bd5726a"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.215375 11992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:54.251814 11992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:54.254523 11992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:54.254660 11992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:54.254868 11992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:54.255221 11992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:54.255259 11992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.255307 11992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:54.255335 11992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.261617 11992 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:45133
I20260812 00:10:54.261667 12248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:45133 every 8 connection(s)
I20260812 00:10:54.262135 11992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:54.267490 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 11992
I20260812 00:10:54.267629 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:10:54.267877 7511 external_mini_cluster.cc:1428] Setting key c2089e6b1300d85ef734315abc0ac915
Aug 12 00:10:54 dist-test-slave-z8x9 krb5kdc[11271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.195@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:10:54.274286 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.263005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:52403 (local address 127.7.85.254:45913)
0812 00:10:54.263100 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.263102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.263309 (+ 207us) server_negotiation.cc:407] Connection header received
0812 00:10:54.263824 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.263826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.263837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.263904 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:54.264943 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.265554 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.266915 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.267316 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.267399 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.270806 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:54.270808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:54.270809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:54.270818 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:54.272699 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:54.273208 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:54.273209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:54.273210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:54.273278 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:54.273555 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:54.273556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:54.273556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:54.273687 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:54.273801 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.274098 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.274187 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:10:54.274525 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.262951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:45913 (local address 127.7.85.195:52403)
0812 00:10:54.263148 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.263194 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.263694 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.263986 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.263990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.264589 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:54.264820 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.264825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.265681 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.265682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.266732 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.266735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.267454 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.267456 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.267666 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.268488 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:10:54.268505 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:54.270661 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:54.272794 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:54.272803 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:54.272805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:54.273069 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:54.273364 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:10:54.273365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:10:54.273365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:10:54.273482 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:10:54.273794 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:54.273802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:54.273949 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.274061 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.274241 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260812 00:10:54.275126 12250 heartbeater.cc:347] Connected to a master server at 127.7.85.254:45913
I20260812 00:10:54.275364 12250 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:54.275739 12250 heartbeater.cc:511] Master 127.7.85.254:45913 requested a full tablet report, sending...
I20260812 00:10:54.276434 11371 ts_manager.cc:194] Registered new tserver with Master: 0b1d4e6415834c068c74ee91d9fef793 (127.7.85.195:45133)
I20260812 00:10:54.276806 11371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.85.195@KRBTEST.COM'} at 127.7.85.195:52403
I20260812 00:10:54.281944 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:54.290778 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.283364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.7.85.254:45913)
0812 00:10:54.283459 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.283461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.283564 (+ 103us) server_negotiation.cc:407] Connection header received
0812 00:10:54.283654 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.283655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.283675 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.283738 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:54.284376 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.284984 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.285857 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.286124 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.286208 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.287822 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:54.287824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:54.287824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:54.287833 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:54.289347 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:54.289726 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:54.289728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:54.289728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:54.289800 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:54.290003 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:54.290004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:54.290004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:54.290178 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:54.290292 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.290476 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.290656 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:10:54.292929 11371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:54.294687 11371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:54.302011 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.297636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:45133 (local address 127.0.0.1:39730)
0812 00:10:54.297824 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.297865 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.297940 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.298324 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.298327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.298329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:54.298429 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.298432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.299390 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.299391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.300813 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.300816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.301665 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.301666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.301752 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.301766 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.301865 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.301926 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":42,"thread_start_us":55,"threads_started":1}
I20260812 00:10:54.302026 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.297728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.7.85.195:45133)
0812 00:10:54.298056 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.298062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.298074 (+ 12us) server_negotiation.cc:407] Connection header received
0812 00:10:54.298134 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.298152 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.298170 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.298450 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:54.298530 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.299269 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.300920 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.301566 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.301662 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.301729 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.301883 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.301924 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":111,"threads_started":1}
I20260812 00:10:54.302119 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.297863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.7.85.193:37141)
0812 00:10:54.298291 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.298296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.298306 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:10:54.298343 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.298345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.298359 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.298436 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:54.298641 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.299360 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.300876 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.301675 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.301748 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.301800 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.301993 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.302033 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":87,"threads_started":1}
I20260812 00:10:54.302374 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.297726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:37141 (local address 127.0.0.1:37018)
0812 00:10:54.298097 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.298108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.298168 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.298510 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.298511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.298513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:54.298551 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.298552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.299481 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.299482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.300763 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.300772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.301836 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.301837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.301908 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.301912 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.302257 (+ 345us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.302303 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260812 00:10:54.303431 11605 tablet_service.cc:1467] Processing CreateTablet for tablet 646fb16216e94476850c474cc0139991 (DEFAULT_TABLE table=test-table [id=1eedfbf1b92e4beb973e9d2446055622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:54.303435 12137 tablet_service.cc:1467] Processing CreateTablet for tablet 646fb16216e94476850c474cc0139991 (DEFAULT_TABLE table=test-table [id=1eedfbf1b92e4beb973e9d2446055622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:54.303828 11605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646fb16216e94476850c474cc0139991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:54.303830 12137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646fb16216e94476850c474cc0139991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:54.307204 12286 tablet_bootstrap.cc:492] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Bootstrap starting.
I20260812 00:10:54.307281 12285 tablet_bootstrap.cc:492] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Bootstrap starting.
I20260812 00:10:54.307816 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.297965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35197 (local address 127.0.0.1:53390)
0812 00:10:54.298193 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.298204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.298317 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.299762 (+ 1445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.299763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.299766 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:54.299816 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.299817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.301047 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.301048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.302135 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.302138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.307489 (+ 5351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.307491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.307584 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.307590 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.307664 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.307737 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":72,"threads_started":1}
I20260812 00:10:54.307864 12284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.298099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.7.85.194:35197)
0812 00:10:54.299566 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.299570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.299579 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:10:54.299618 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.299621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.299648 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.299733 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:54.299911 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.300950 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.302230 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.302930 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.307355 (+ 4425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.307452 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.307708 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.307775 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":99,"threads_started":1}
I20260812 00:10:54.308053 12285 tablet_bootstrap.cc:654] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:54.308525 12286 tablet_bootstrap.cc:654] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:54.308745 12285 log.cc:824] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:54.308661 11862 tablet_service.cc:1467] Processing CreateTablet for tablet 646fb16216e94476850c474cc0139991 (DEFAULT_TABLE table=test-table [id=1eedfbf1b92e4beb973e9d2446055622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:54.309039 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 646fb16216e94476850c474cc0139991. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:54.309206 12286 log.cc:824] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:54.309863 12285 tablet_bootstrap.cc:492] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: No bootstrap required, opened a new log
I20260812 00:10:54.309942 12285 ts_tablet_manager.cc:1397] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:10:54.312423 12285 raft_consensus.cc:348] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.312667 12285 raft_consensus.cc:374] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:54.312717 12285 raft_consensus.cc:729] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1d4e6415834c068c74ee91d9fef793, State: Initialized, Role: FOLLOWER
I20260812 00:10:54.312852 12285 consensus_queue.cc:260] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.313215 12285 ts_tablet_manager.cc:1428] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 00:10:54.313380 12250 heartbeater.cc:503] Master 127.7.85.254:45913 was elected leader, sending a full tablet report...
I20260812 00:10:54.313632 12286 tablet_bootstrap.cc:492] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: No bootstrap required, opened a new log
I20260812 00:10:54.313719 12286 ts_tablet_manager.cc:1397] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260812 00:10:54.315511 12286 raft_consensus.cc:348] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.315665 12286 raft_consensus.cc:374] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:54.315686 12286 raft_consensus.cc:729] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f281b408a5544d1a5206cbcc44d8e2d, State: Initialized, Role: FOLLOWER
I20260812 00:10:54.315814 12286 consensus_queue.cc:260] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.316218 12286 ts_tablet_manager.cc:1428] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 00:10:54.316258 11715 heartbeater.cc:503] Master 127.7.85.254:45913 was elected leader, sending a full tablet report...
I20260812 00:10:54.316527 12298 tablet_bootstrap.cc:492] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Bootstrap starting.
I20260812 00:10:54.317364 12298 tablet_bootstrap.cc:654] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:54.318099 12298 log.cc:824] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:54.319084 12298 tablet_bootstrap.cc:492] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: No bootstrap required, opened a new log
I20260812 00:10:54.319168 12298 ts_tablet_manager.cc:1397] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 00:10:54.320961 12298 raft_consensus.cc:348] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.321141 12298 raft_consensus.cc:374] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:54.321220 12298 raft_consensus.cc:729] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8553ef067904111879ffa1bbc5de705, State: Initialized, Role: FOLLOWER
I20260812 00:10:54.321379 12298 consensus_queue.cc:260] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.321697 12298 ts_tablet_manager.cc:1428] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 00:10:54.321798 11977 heartbeater.cc:503] Master 127.7.85.254:45913 was elected leader, sending a full tablet report...
W20260812 00:10:54.343752 11716 tablet.cc:2369] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:10:54.377596 12299 raft_consensus.cc:482] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:54.377763 12299 raft_consensus.cc:504] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.378196 12299 leader_election.cc:288] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f281b408a5544d1a5206cbcc44d8e2d (127.7.85.193:37141), d8553ef067904111879ffa1bbc5de705 (127.7.85.194:35197)
I20260812 00:10:54.381624 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.378588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35197 (local address 127.7.85.195:54715)
0812 00:10:54.378777 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.378789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.378848 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.378997 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.378998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.378999 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:54.379066 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.379067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.379781 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.379782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.380628 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.380631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.381353 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.381355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.381434 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.381440 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.381500 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.381551 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":127,"threads_started":1}
I20260812 00:10:54.381673 12284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.378716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:54715 (local address 127.7.85.194:35197)
0812 00:10:54.378772 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.378774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.378859 (+ 85us) server_negotiation.cc:407] Connection header received
0812 00:10:54.378914 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.378915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.378928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.378981 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:54.379138 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.379631 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.380750 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.381162 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.381278 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.381294 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.381531 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.381579 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 00:10:54.381850 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.378551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:35341 (local address 127.7.85.193:37141)
0812 00:10:54.378624 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.378626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.378639 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:10:54.378674 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.378675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.378689 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.378781 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:54.378986 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.379533 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.380849 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.381390 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.381467 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.381477 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.381711 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.381762 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:10:54.381889 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.378448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:37141 (local address 127.7.85.195:35341)
0812 00:10:54.378512 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.378526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.378576 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.378791 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.378792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.378794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:54.378898 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.378901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.379974 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.379975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.380719 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.380722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.381514 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.381516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.381608 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.381612 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.381682 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.381808 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:10:54.382139 11900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646fb16216e94476850c474cc0139991" candidate_uuid: "0b1d4e6415834c068c74ee91d9fef793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8553ef067904111879ffa1bbc5de705" is_pre_election: true
I20260812 00:10:54.382265 11638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646fb16216e94476850c474cc0139991" candidate_uuid: "0b1d4e6415834c068c74ee91d9fef793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" is_pre_election: true
I20260812 00:10:54.382401 11900 raft_consensus.cc:2454] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1d4e6415834c068c74ee91d9fef793 in term 0.
I20260812 00:10:54.382448 11638 raft_consensus.cc:2454] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b1d4e6415834c068c74ee91d9fef793 in term 0.
I20260812 00:10:54.382668 12032 leader_election.cc:302] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1d4e6415834c068c74ee91d9fef793, d8553ef067904111879ffa1bbc5de705; no voters:
I20260812 00:10:54.382858 12299 raft_consensus.cc:2781] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:54.382930 12299 raft_consensus.cc:482] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:54.382947 12299 raft_consensus.cc:3037] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:54.383832 12299 raft_consensus.cc:504] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.383955 12299 leader_election.cc:288] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [CANDIDATE]: Term 1 election: Requested vote from peers 8f281b408a5544d1a5206cbcc44d8e2d (127.7.85.193:37141), d8553ef067904111879ffa1bbc5de705 (127.7.85.194:35197)
I20260812 00:10:54.384214 11638 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646fb16216e94476850c474cc0139991" candidate_uuid: "0b1d4e6415834c068c74ee91d9fef793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f281b408a5544d1a5206cbcc44d8e2d"
I20260812 00:10:54.384353 11638 raft_consensus.cc:3037] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:54.385288 11638 raft_consensus.cc:2454] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1d4e6415834c068c74ee91d9fef793 in term 1.
I20260812 00:10:54.385392 11900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "646fb16216e94476850c474cc0139991" candidate_uuid: "0b1d4e6415834c068c74ee91d9fef793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8553ef067904111879ffa1bbc5de705"
I20260812 00:10:54.385515 11900 raft_consensus.cc:3037] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:54.385540 12032 leader_election.cc:302] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b1d4e6415834c068c74ee91d9fef793, 8f281b408a5544d1a5206cbcc44d8e2d; no voters:
I20260812 00:10:54.386476 11900 raft_consensus.cc:2454] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b1d4e6415834c068c74ee91d9fef793 in term 1.
I20260812 00:10:54.386603 12299 raft_consensus.cc:2781] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:54.386797 12299 raft_consensus.cc:686] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1d4e6415834c068c74ee91d9fef793, State: Running, Role: LEADER
I20260812 00:10:54.386916 12299 consensus_queue.cc:237] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } }
I20260812 00:10:54.387853 11365 catalog_manager.cc:5495] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b1d4e6415834c068c74ee91d9fef793 (127.7.85.195). New cstate: current_term: 1 leader_uuid: "0b1d4e6415834c068c74ee91d9fef793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b1d4e6415834c068c74ee91d9fef793" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 45133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 } health_report { overall_health: UNKNOWN } } }
I20260812 00:10:54.396358 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.393935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39736 (local address 127.7.85.195:45133)
0812 00:10:54.394004 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.394007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.394015 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:10:54.394054 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.394055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.394083 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.394164 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:54.394282 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.394765 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.395544 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.395765 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.395848 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.396003 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:54.396116 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:54.396214 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.396233 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.396255 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 00:10:54.398731 11900 raft_consensus.cc:1261] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 1 FOLLOWER]: Refusing update from remote peer 0b1d4e6415834c068c74ee91d9fef793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:54.398933 11638 raft_consensus.cc:1261] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 1 FOLLOWER]: Refusing update from remote peer 0b1d4e6415834c068c74ee91d9fef793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:54.399263 12299 consensus_queue.cc:1035] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8553ef067904111879ffa1bbc5de705" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:54.399434 12311 consensus_queue.cc:1035] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f281b408a5544d1a5206cbcc44d8e2d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 37141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:54.403121 12316 mvcc.cc:205] Tried to move back new op lower bound from 7317477189213896704 to 7317477189169651712. Current Snapshot: MvccSnapshot[applied={T|T < 7317477189213896704}]
I20260812 00:10:54.403381 12317 mvcc.cc:205] Tried to move back new op lower bound from 7317477189213896704 to 7317477189169651712. Current Snapshot: MvccSnapshot[applied={T|T < 7317477189213896704}]
I20260812 00:10:54.404803 11365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:54.404932 11365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:54.405850 11365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abf610ebc4574cdca56a499984cc5ec1: Sending DeleteTablet for 3 replicas of tablet 646fb16216e94476850c474cc0139991
I20260812 00:10:54.406213 11605 tablet_service.cc:1514] Processing DeleteTablet for tablet 646fb16216e94476850c474cc0139991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:54 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:37018
I20260812 00:10:54.406246 11862 tablet_service.cc:1514] Processing DeleteTablet for tablet 646fb16216e94476850c474cc0139991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:54 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:53390
I20260812 00:10:54.406524 12330 tablet_replica.cc:330] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: stopping tablet replica
I20260812 00:10:54.406477 12137 tablet_service.cc:1514] Processing DeleteTablet for tablet 646fb16216e94476850c474cc0139991 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:54 UTC) from {username='kudu', principal='kudu/127.7.85.254@KRBTEST.COM'} at 127.0.0.1:39730
I20260812 00:10:54.406646 12330 raft_consensus.cc:2229] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:54.406764 12330 raft_consensus.cc:2258] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:54.407219 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11494
I20260812 00:10:54.407236 12330 ts_tablet_manager.cc:1905] T 646fb16216e94476850c474cc0139991 P 8f281b408a5544d1a5206cbcc44d8e2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:54.407794 12332 tablet_replica.cc:330] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: stopping tablet replica
I20260812 00:10:54.407909 12332 raft_consensus.cc:2229] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:54.408020 12332 raft_consensus.cc:2258] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:54.418156 12334 tablet_replica.cc:330] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: stopping tablet replica
I20260812 00:10:54.418526 12334 raft_consensus.cc:2229] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:10:54.418773 12334 raft_consensus.cc:2258] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:54.421330 12334 ts_tablet_manager.cc:1905] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:54.423841 12334 ts_tablet_manager.cc:1918] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:54.423945 12334 log.cc:1196] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/646fb16216e94476850c474cc0139991
I20260812 00:10:54.424259 12334 ts_tablet_manager.cc:1939] T 646fb16216e94476850c474cc0139991 P 0b1d4e6415834c068c74ee91d9fef793: Deleting consensus metadata
I20260812 00:10:54.424830 11343 catalog_manager.cc:4841] TS 0b1d4e6415834c068c74ee91d9fef793 (127.7.85.195:45133): tablet 646fb16216e94476850c474cc0139991 (table test-table [id=1eedfbf1b92e4beb973e9d2446055622]) successfully deleted
I20260812 00:10:54.425916 12332 ts_tablet_manager.cc:1905] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:54.428030 12332 ts_tablet_manager.cc:1918] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:54.428216 12332 log.cc:1196] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/646fb16216e94476850c474cc0139991
W20260812 00:10:54.428520 11337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:37141 (error 108)
I20260812 00:10:54.428637 12332 ts_tablet_manager.cc:1939] T 646fb16216e94476850c474cc0139991 P d8553ef067904111879ffa1bbc5de705: Deleting consensus metadata
W20260812 00:10:54.429131 12282 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:10:54.428964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:37141 (local address 127.0.0.1:37028)
0812 00:10:54.429023 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.429082 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:37141: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 00:10:54.429347 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11725
W20260812 00:10:54.429443 11337 catalog_manager.cc:4584] TS 8f281b408a5544d1a5206cbcc44d8e2d (127.7.85.193:37141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 646fb16216e94476850c474cc0139991: Network error: Client connection negotiation failed: client connection to 127.7.85.193:37141: connect: Connection reset by peer (error 104)
I20260812 00:10:54.450091 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11992
I20260812 00:10:54.450078 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.449921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35197 (local address 127.0.0.1:53392)
0812 00:10:54.449984 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.450019 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.194:35197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 00:10:54.459692 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 11300
2026-08-12T00:10:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2542 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T00:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:54Z Disabled control of system clock
I20260812 00:10:54.498983 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:45201
--webserver_interface=127.7.85.254
--webserver_port=0
--builtin_ntp_servers=127.7.85.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:45201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:54.589449 12342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:54.589692 12342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:54.589715 12342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:54.590865 12342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:54.590905 12342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:54.590919 12342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:54.590931 12342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:54.590943 12342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:54.592499 12342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:45201
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:45201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:54.592759 12342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:54.593036 12342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:54.598793 12356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:54.598867 12362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:54.598853 12353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:54.599205 12342 server_base.cc:1034] running on GCE node
I20260812 00:10:54.599509 12342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:54.599826 12342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:54.601023 12342 hybrid_clock.cc:648] HybridClock initialized: now 1786493454601011 us; error 33 us; skew 500 ppm
I20260812 00:10:54.602516 12342 webserver.cc:460] Webserver started at http://127.7.85.254:34859/ using document root <none> and password file <none>
I20260812 00:10:54.602813 12342 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:54.602892 12342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:54.603068 12342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:54.604370 12342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "4fffbd23667749e581542893b08672ce"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "02172f868823a256d86ea1622b8ca277"
server_key_iv: "bad9dfa2e62222c2af311dc076fac2d0"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.604779 12342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "4fffbd23667749e581542893b08672ce"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "02172f868823a256d86ea1622b8ca277"
server_key_iv: "bad9dfa2e62222c2af311dc076fac2d0"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.606444 12342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:10:54.607398 12378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.607693 12342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:54.607802 12342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "4fffbd23667749e581542893b08672ce"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "02172f868823a256d86ea1622b8ca277"
server_key_iv: "bad9dfa2e62222c2af311dc076fac2d0"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.607896 12342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:54.632002 12342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:54.632398 12342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:54.632565 12342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:54.637357 12342 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:45201
I20260812 00:10:54.637434 12492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:45201 every 8 connection(s)
I20260812 00:10:54.637836 12342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:54.639147 12494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:54.641623 12494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce: Bootstrap starting.
I20260812 00:10:54.642304 12494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:54.642632 12494 log.cc:824] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:54.643433 12494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce: No bootstrap required, opened a new log
I20260812 00:10:54.645308 12494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fffbd23667749e581542893b08672ce" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45201 } }
I20260812 00:10:54.645462 12494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:54.645486 12494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fffbd23667749e581542893b08672ce, State: Initialized, Role: FOLLOWER
I20260812 00:10:54.645648 12494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fffbd23667749e581542893b08672ce" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45201 } }
I20260812 00:10:54.645726 12494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:54.645749 12494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:54.645787 12494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:54.646008 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12342
I20260812 00:10:54.646143 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:54.646377 7511 external_mini_cluster.cc:1428] Setting key 283d05aca209887cf2448b4801a6885d
I20260812 00:10:54.646525 12494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fffbd23667749e581542893b08672ce" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45201 } }
I20260812 00:10:54.646670 12494 leader_election.cc:302] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fffbd23667749e581542893b08672ce; no voters:
I20260812 00:10:54.646912 12494 leader_election.cc:288] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:54.646961 12500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:54.647115 12500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [term 1 LEADER]: Becoming Leader. State: Replica: 4fffbd23667749e581542893b08672ce, State: Running, Role: LEADER
I20260812 00:10:54.647230 12494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:54.647207 12500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fffbd23667749e581542893b08672ce" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45201 } }
I20260812 00:10:54.647616 12502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fffbd23667749e581542893b08672ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fffbd23667749e581542893b08672ce" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45201 } } }
I20260812 00:10:54.647723 12506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fffbd23667749e581542893b08672ce. Latest consensus state: current_term: 1 leader_uuid: "4fffbd23667749e581542893b08672ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fffbd23667749e581542893b08672ce" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 45201 } } }
I20260812 00:10:54.647835 12506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:54.647727 12502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:54.648140 12515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:54.649096 12515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:54.653131 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.646971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.7.85.254:45201)
0812 00:10:54.647413 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.647422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.647459 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:10:54.647587 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.647595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.647669 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.648283 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:54.648400 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.649944 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.651064 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.651522 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.651628 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.652405 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:54.652409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:54.652409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:54.652584 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:54.652616 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:54.652627 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:54.652681 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.652786 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.652893 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":110,"threads_started":1}
I20260812 00:10:54.654393 12515 catalog_manager.cc:1347] Generated new cluster ID: c54b28483a1c4b0fa35274a22da99d54
I20260812 00:10:54.654456 12515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:54.670588 12515 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:54.671384 12515 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:54.674000 12515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce: Generated new TSK 0
I20260812 00:10:54.674209 12515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:10:54.680543 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.193:0
--local_ip_for_outbound_sockets=127.7.85.193
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45201
--builtin_ntp_servers=127.7.85.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:10:54.773979 12538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:54.774255 12538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:54.774295 12538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:54.775499 12538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:54.775570 12538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:54.775616 12538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.193
I20260812 00:10:54.777163 12538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.85.193
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.193
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:54.777498 12538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:54.777829 12538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:54.784426 12551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:54.784531 12538 server_base.cc:1034] running on GCE node
W20260812 00:10:54.784636 12552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:54.784437 12557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:54.784875 12538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:54.785260 12538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:54.786474 12538 hybrid_clock.cc:648] HybridClock initialized: now 1786493454786443 us; error 47 us; skew 500 ppm
I20260812 00:10:54.788096 12538 webserver.cc:460] Webserver started at http://127.7.85.193:44131/ using document root <none> and password file <none>
I20260812 00:10:54.788374 12538 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:54.788437 12538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:54.788565 12538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:54.790073 12538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data/instance:
uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "d1741629cc1a0c33bce96102909f479f"
server_key_iv: "a6431f1a278b0b67e2e26aed3b675e69"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.790477 12538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance:
uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "d1741629cc1a0c33bce96102909f479f"
server_key_iv: "a6431f1a278b0b67e2e26aed3b675e69"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.792436 12538 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:10:54.793658 12569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.793886 12538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:54.793987 12538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal
uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "d1741629cc1a0c33bce96102909f479f"
server_key_iv: "a6431f1a278b0b67e2e26aed3b675e69"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.794063 12538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:54.812088 12538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:54.812435 12538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:54.812562 12538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:54.812793 12538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:54.813252 12538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:54.813290 12538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.813318 12538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:54.813372 12538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.821367 12538 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.193:38109
I20260812 00:10:54.821462 12809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.193:38109 every 8 connection(s)
I20260812 00:10:54.821825 12538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/data/info.pb
I20260812 00:10:54.827702 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12538
I20260812 00:10:54.827862 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-0/wal/instance
I20260812 00:10:54.827852 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.822767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.193:46965 (local address 127.7.85.254:45201)
0812 00:10:54.822829 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.822831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.823064 (+ 233us) server_negotiation.cc:407] Connection header received
0812 00:10:54.823596 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.823597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.823611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.823686 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:54.824521 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.825050 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.826256 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.826460 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.826527 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.827487 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:54.827489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:54.827489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:54.827495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:54.827514 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:54.827520 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:54.827590 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.827703 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.827745 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 00:10:54.828065 7511 external_mini_cluster.cc:1428] Setting key fb5e3c03e630261996c34b28bab56db5
I20260812 00:10:54.828058 12818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.822661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:45201 (local address 127.7.85.193:46965)
0812 00:10:54.822884 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.822934 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.823478 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.823787 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.823791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.823800 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:54.824377 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.824382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.825163 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.825164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.826123 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.826126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.826565 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.826566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.826694 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.827298 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:54.827298 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:54.827338 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:54.827364 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:54.827601 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:54.827607 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:54.827610 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.827674 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.827768 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
I20260812 00:10:54.828644 12814 heartbeater.cc:347] Connected to a master server at 127.7.85.254:45201
I20260812 00:10:54.828799 12814 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:54.829056 12814 heartbeater.cc:511] Master 127.7.85.254:45201 requested a full tablet report, sending...
I20260812 00:10:54.829438 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.194:0
--local_ip_for_outbound_sockets=127.7.85.194
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45201
--builtin_ntp_servers=127.7.85.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:10:54.829713 12416 ts_manager.cc:194] Registered new tserver with Master: 41afdfee8bbc4e8bbb1e22ebf3344e5d (127.7.85.193:38109)
I20260812 00:10:54.830356 12416 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.193:46965
W20260812 00:10:54.924497 12825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:54.924755 12825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:54.924779 12825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:54.926155 12825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:54.926216 12825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:54.926245 12825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.194
I20260812 00:10:54.928135 12825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.85.194
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.194
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:54.928471 12825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:54.928848 12825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:54.934805 12834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:54.934794 12838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:54.934794 12835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:54.934830 12825 server_base.cc:1034] running on GCE node
I20260812 00:10:54.935281 12825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:54.935611 12825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:54.936815 12825 hybrid_clock.cc:648] HybridClock initialized: now 1786493454936793 us; error 44 us; skew 500 ppm
I20260812 00:10:54.938397 12825 webserver.cc:460] Webserver started at http://127.7.85.194:39971/ using document root <none> and password file <none>
I20260812 00:10:54.938681 12825 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:54.938762 12825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:54.938930 12825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:54.940059 12825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data/instance:
uuid: "85ac9013e1ca48a1bbaf26e75b23d956"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "167446a26d6bccb9bd0747611f695a7f"
server_key_iv: "2ea32106d4f7e13c0c9b1b383a011787"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.940443 12825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance:
uuid: "85ac9013e1ca48a1bbaf26e75b23d956"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "167446a26d6bccb9bd0747611f695a7f"
server_key_iv: "2ea32106d4f7e13c0c9b1b383a011787"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.942260 12825 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 00:10:54.943241 12850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.943466 12825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:54.943583 12825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal
uuid: "85ac9013e1ca48a1bbaf26e75b23d956"
format_stamp: "Formatted at 2026-08-12 00:10:54 on dist-test-slave-z8x9"
server_key: "167446a26d6bccb9bd0747611f695a7f"
server_key_iv: "2ea32106d4f7e13c0c9b1b383a011787"
server_key_version: "encryptionkey@0"
I20260812 00:10:54.943663 12825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:54.956766 12825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:54.957206 12825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:54.957405 12825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:54.957697 12825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:54.958084 12825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:54.958146 12825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.958195 12825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:54.958230 12825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:54.965402 12825 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.194:35539
I20260812 00:10:54.965471 13074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.194:35539 every 8 connection(s)
I20260812 00:10:54.965864 12825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/data/info.pb
I20260812 00:10:54.966583 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 12825
I20260812 00:10:54.966746 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal/instance
I20260812 00:10:54.967029 7511 external_mini_cluster.cc:1428] Setting key 3c5e6c884741e693972d6d4b35437055
I20260812 00:10:54.968446 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.85.195:0
--local_ip_for_outbound_sockets=127.7.85.195
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45201
--builtin_ntp_servers=127.7.85.212:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:10:54.972530 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.967674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:39915 (local address 127.7.85.254:45201)
0812 00:10:54.967719 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:54.967722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:54.967732 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:10:54.968392 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:54.968393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:54.968408 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:54.968461 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:54.969299 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.969809 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.970837 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.971061 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.971156 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.972230 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:54.972231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:54.972231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:54.972237 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:54.972252 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:54.972256 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:54.972318 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:54.972439 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 00:10:54.972464 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:10:54.972797 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:54.967301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:45201 (local address 127.7.85.194:39915)
0812 00:10:54.967581 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:54.967628 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:54.968281 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:54.968549 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:54.968553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:54.968587 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:54.969162 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.969215 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.969912 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.969913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.970739 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:54.970742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:54.971170 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:54.971171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:54.971362 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:54.972019 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:54.972020 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:54.972061 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:54.972094 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:54.972335 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:54.972342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:54.972345 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:54.972419 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 00:10:54.972545 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":131,"threads_started":1}
I20260812 00:10:54.973443 13075 heartbeater.cc:347] Connected to a master server at 127.7.85.254:45201
I20260812 00:10:54.973591 13075 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:54.973830 13075 heartbeater.cc:511] Master 127.7.85.254:45201 requested a full tablet report, sending...
I20260812 00:10:54.974273 12416 ts_manager.cc:194] Registered new tserver with Master: 85ac9013e1ca48a1bbaf26e75b23d956 (127.7.85.194:35539)
I20260812 00:10:54.974838 12416 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.194:39915
W20260812 00:10:55.057472 13083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:55.057717 13083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:55.057744 13083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:55.058894 13083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:55.058931 13083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:55.058956 13083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.85.195
I20260812 00:10:55.060283 13083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.85.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.85.195
--webserver_port=0
--tserver_master_addrs=127.7.85.254:45201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.85.195
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:55.060494 13083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:55.060768 13083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:55.066100 13091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:55.066174 13083 server_base.cc:1034] running on GCE node
W20260812 00:10:55.066109 13092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:55.066109 13096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:55.066771 13083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:55.067082 13083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:55.068485 13083 hybrid_clock.cc:648] HybridClock initialized: now 1786493455068533 us; error 99 us; skew 500 ppm
I20260812 00:10:55.070035 13083 webserver.cc:460] Webserver started at http://127.7.85.195:44917/ using document root <none> and password file <none>
I20260812 00:10:55.070297 13083 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:55.070382 13083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:55.070538 13083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:55.071650 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data/instance:
uuid: "8564b462de9248c5936842df896c60b0"
format_stamp: "Formatted at 2026-08-12 00:10:55 on dist-test-slave-z8x9"
server_key: "27cf6d57880f66c33e2b468c10f6525d"
server_key_iv: "49f76b51a40d45cfeec0483ac36d11bf"
server_key_version: "encryptionkey@0"
I20260812 00:10:55.072026 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance:
uuid: "8564b462de9248c5936842df896c60b0"
format_stamp: "Formatted at 2026-08-12 00:10:55 on dist-test-slave-z8x9"
server_key: "27cf6d57880f66c33e2b468c10f6525d"
server_key_iv: "49f76b51a40d45cfeec0483ac36d11bf"
server_key_version: "encryptionkey@0"
I20260812 00:10:55.073608 13083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260812 00:10:55.074468 13108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:55.074678 13083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:55.074790 13083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal
uuid: "8564b462de9248c5936842df896c60b0"
format_stamp: "Formatted at 2026-08-12 00:10:55 on dist-test-slave-z8x9"
server_key: "27cf6d57880f66c33e2b468c10f6525d"
server_key_iv: "49f76b51a40d45cfeec0483ac36d11bf"
server_key_version: "encryptionkey@0"
I20260812 00:10:55.074883 13083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:55.089439 13083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:55.089813 13083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:55.089986 13083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:55.090274 13083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:10:55.090662 13083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:10:55.090727 13083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:55.090773 13083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:10:55.090816 13083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:55.097095 13083 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.195:42885
I20260812 00:10:55.097190 13363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.195:42885 every 8 connection(s)
I20260812 00:10:55.097586 13083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/data/info.pb
I20260812 00:10:55.103420 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.098491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.195:51453 (local address 127.7.85.254:45201)
0812 00:10:55.098565 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.098568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.098812 (+ 244us) server_negotiation.cc:407] Connection header received
0812 00:10:55.099410 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.099411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.099425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.099487 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:55.100258 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.100783 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.101832 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.102038 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.102127 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.103080 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:55.103081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:55.103082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:55.103087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:55.103102 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:55.103107 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:55.103176 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.103293 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.103330 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:10:55.103670 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.098449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:45201 (local address 127.7.85.195:51453)
0812 00:10:55.098665 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.098713 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:55.099291 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:55.099561 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:55.099570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:55.099579 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:10:55.100138 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.100143 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.100888 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.100890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.101731 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.101733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.102143 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.102144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.102294 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.102922 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:10:55.102923 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:10:55.102953 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:10:55.102979 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:10:55.103190 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:10:55.103198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:10:55.103201 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 00:10:55.103295 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 00:10:55.103407 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260812 00:10:55.104317 13365 heartbeater.cc:347] Connected to a master server at 127.7.85.254:45201
I20260812 00:10:55.104595 13365 heartbeater.cc:464] Registering TS with master...
I20260812 00:10:55.104832 13365 heartbeater.cc:511] Master 127.7.85.254:45201 requested a full tablet report, sending...
I20260812 00:10:55.105388 12416 ts_manager.cc:194] Registered new tserver with Master: 8564b462de9248c5936842df896c60b0 (127.7.85.195:42885)
I20260812 00:10:55.105973 12416 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.85.195:51453
I20260812 00:10:55.106169 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13083
I20260812 00:10:55.106416 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal/instance
I20260812 00:10:55.106704 7511 external_mini_cluster.cc:1428] Setting key 0de5477da2254ce914016ca63adc7877
I20260812 00:10:55.107805 7511 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:10:55.112572 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.109317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45966 (local address 127.7.85.254:45201)
0812 00:10:55.109373 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.109376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.109510 (+ 134us) server_negotiation.cc:407] Connection header received
0812 00:10:55.109589 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.109591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.109603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.109648 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:55.109848 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.110334 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.111162 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.111378 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.111457 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.112295 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:55.112296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:55.112297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:55.112304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:55.112323 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:55.112327 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:55.112391 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.112456 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.112481 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 00:10:55.115302 12416 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:10:55.116950 12416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:10:55.123739 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.119465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38109 (local address 127.0.0.1:37002)
0812 00:10:55.119641 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.119680 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:55.119752 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:55.120245 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:55.120248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:55.120251 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:55.120333 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.120335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.121323 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.121324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.122541 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.122544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.123428 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.123429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.123510 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.123525 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:10:55.123608 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 00:10:55.123673 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260812 00:10:55.123906 13385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.119465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:42885 (local address 127.0.0.1:40070)
0812 00:10:55.119810 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.119819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:55.119859 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:55.120406 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:55.120407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:55.120409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:55.120441 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.120442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.121766 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.121767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.122768 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.122771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.123638 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.123640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.123709 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.123713 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 00:10:55.123765 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 00:10:55.123820 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":4,"thread_start_us":61,"threads_started":1}
I20260812 00:10:55.123909 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.119839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.7.85.195:42885)
0812 00:10:55.120042 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.120048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.120061 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:10:55.120105 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.120107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.120124 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.120226 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:55.120522 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.121666 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.122854 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.123548 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.123626 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.123716 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.123776 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.123812 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260812 00:10:55.125023 13384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.119599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.7.85.193:38109)
0812 00:10:55.120044 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.120048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.120061 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:10:55.120105 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.120107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.120124 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.120349 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:55.120411 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.121227 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.122626 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.123332 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.123414 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.123511 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.124867 (+ 1356us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.124921 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":146,"threads_started":1}
I20260812 00:10:55.124905 13215 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0f2c81dde64f439b6002b84be66a41 (DEFAULT_TABLE table=test-table [id=f27eb4bae1794408ac3d2cc27f5da656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:55.125079 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.119668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.194:35539 (local address 127.0.0.1:52812)
0812 00:10:55.119952 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.119961 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:55.119997 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:55.120359 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:55.120360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:55.120362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:55.120400 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.120402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.122839 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.122839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.123545 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.123547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.124798 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.124801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.124869 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.124875 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:10:55.124956 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 00:10:55.125011 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":92,"threads_started":1}
I20260812 00:10:55.125402 13215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0f2c81dde64f439b6002b84be66a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:55.125537 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.119799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52812 (local address 127.7.85.194:35539)
0812 00:10:55.120012 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.120098 (+ 86us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.120115 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:10:55.120190 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.120192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.120211 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.120294 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:55.120477 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.121675 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.124093 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.124624 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.124723 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.124817 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.125396 (+ 579us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.125450 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260812 00:10:55.125694 12692 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0f2c81dde64f439b6002b84be66a41 (DEFAULT_TABLE table=test-table [id=f27eb4bae1794408ac3d2cc27f5da656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:55.126009 12692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0f2c81dde64f439b6002b84be66a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:55.126233 12944 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0f2c81dde64f439b6002b84be66a41 (DEFAULT_TABLE table=test-table [id=f27eb4bae1794408ac3d2cc27f5da656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:10:55.126552 12944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0f2c81dde64f439b6002b84be66a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:55.128718 13390 tablet_bootstrap.cc:492] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: Bootstrap starting.
I20260812 00:10:55.129081 13391 tablet_bootstrap.cc:492] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Bootstrap starting.
I20260812 00:10:55.129495 13390 tablet_bootstrap.cc:654] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:55.129770 13391 tablet_bootstrap.cc:654] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:55.129866 13390 log.cc:824] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:55.130107 13391 log.cc:824] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:55.130467 13392 tablet_bootstrap.cc:492] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Bootstrap starting.
I20260812 00:10:55.130774 13390 tablet_bootstrap.cc:492] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: No bootstrap required, opened a new log
I20260812 00:10:55.130851 13390 ts_tablet_manager.cc:1397] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:10:55.131011 13391 tablet_bootstrap.cc:492] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: No bootstrap required, opened a new log
I20260812 00:10:55.131065 13391 ts_tablet_manager.cc:1397] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:10:55.131119 13392 tablet_bootstrap.cc:654] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:55.131462 13392 log.cc:824] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:55.132154 13392 tablet_bootstrap.cc:492] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: No bootstrap required, opened a new log
I20260812 00:10:55.132202 13392 ts_tablet_manager.cc:1397] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:10:55.134199 13392 raft_consensus.cc:348] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.134341 13392 raft_consensus.cc:374] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:55.134363 13392 raft_consensus.cc:729] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ac9013e1ca48a1bbaf26e75b23d956, State: Initialized, Role: FOLLOWER
I20260812 00:10:55.134354 13390 raft_consensus.cc:348] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.134495 13392 consensus_queue.cc:260] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.134549 13390 raft_consensus.cc:374] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:55.134586 13390 raft_consensus.cc:729] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41afdfee8bbc4e8bbb1e22ebf3344e5d, State: Initialized, Role: FOLLOWER
I20260812 00:10:55.134485 13391 raft_consensus.cc:348] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.134627 13391 raft_consensus.cc:374] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:55.134671 13391 raft_consensus.cc:729] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8564b462de9248c5936842df896c60b0, State: Initialized, Role: FOLLOWER
I20260812 00:10:55.134748 13390 consensus_queue.cc:260] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.134784 13391 consensus_queue.cc:260] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.134831 13392 ts_tablet_manager.cc:1428] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:10:55.134953 13075 heartbeater.cc:503] Master 127.7.85.254:45201 was elected leader, sending a full tablet report...
I20260812 00:10:55.135051 13391 ts_tablet_manager.cc:1428] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 00:10:55.135051 13390 ts_tablet_manager.cc:1428] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 00:10:55.135177 12814 heartbeater.cc:503] Master 127.7.85.254:45201 was elected leader, sending a full tablet report...
I20260812 00:10:55.135191 13365 heartbeater.cc:503] Master 127.7.85.254:45201 was elected leader, sending a full tablet report...
W20260812 00:10:55.216759 13077 tablet.cc:2369] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:10:55.243553 13397 raft_consensus.cc:482] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:10:55.243786 13397 raft_consensus.cc:504] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.244424 13397 leader_election.cc:288] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41afdfee8bbc4e8bbb1e22ebf3344e5d (127.7.85.193:38109), 8564b462de9248c5936842df896c60b0 (127.7.85.195:42885)
I20260812 00:10:55.249408 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.244740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.195:42885 (local address 127.7.85.194:50643)
0812 00:10:55.244829 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.244845 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:55.244911 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:55.245280 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:55.245281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:55.245284 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:55.245405 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.245408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.246511 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.246511 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.248221 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.248229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.249054 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.249056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.249135 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.249141 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:10:55.249216 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 00:10:55.249294 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":56}
I20260812 00:10:55.249497 13384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.244885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:47099 (local address 127.7.85.193:38109)
0812 00:10:55.244968 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.244971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.245450 (+ 479us) server_negotiation.cc:407] Connection header received
0812 00:10:55.245526 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.245528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.245543 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.245599 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:55.245845 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.246592 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.248441 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.248847 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.248934 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.248941 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.249369 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.249420 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:10:55.249482 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.244882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.85.194:50643 (local address 127.7.85.195:42885)
0812 00:10:55.244981 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.244983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.244996 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:10:55.245042 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.245043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.245054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.245149 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:10:55.245579 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.246429 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.248417 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.248897 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.249013 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.249026 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.249302 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.249394 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:10:55.249703 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.244753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38109 (local address 127.7.85.194:47099)
0812 00:10:55.245361 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.245376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:10:55.245451 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:10:55.245608 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:10:55.245609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:10:55.245611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:10:55.245699 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.245701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.246644 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.246645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.248221 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:10:55.248229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.248980 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.248982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:10:55.249073 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.249079 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:10:55.249360 (+ 281us) client_negotiation.cc:240] Negotiation successful
0812 00:10:55.249421 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":229,"thread_start_us":169,"threads_started":1}
I20260812 00:10:55.250203 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0f2c81dde64f439b6002b84be66a41" candidate_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8564b462de9248c5936842df896c60b0" is_pre_election: true
I20260812 00:10:55.250425 13263 raft_consensus.cc:2454] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85ac9013e1ca48a1bbaf26e75b23d956 in term 0.
I20260812 00:10:55.250536 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0f2c81dde64f439b6002b84be66a41" candidate_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" is_pre_election: true
I20260812 00:10:55.250713 12722 raft_consensus.cc:2454] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85ac9013e1ca48a1bbaf26e75b23d956 in term 0.
I20260812 00:10:55.250756 12856 leader_election.cc:302] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8564b462de9248c5936842df896c60b0, 85ac9013e1ca48a1bbaf26e75b23d956; no voters:
I20260812 00:10:55.250960 13397 raft_consensus.cc:2781] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:10:55.251032 13397 raft_consensus.cc:482] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:10:55.251065 13397 raft_consensus.cc:3037] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:55.251876 13397 raft_consensus.cc:504] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.252063 13397 leader_election.cc:288] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [CANDIDATE]: Term 1 election: Requested vote from peers 41afdfee8bbc4e8bbb1e22ebf3344e5d (127.7.85.193:38109), 8564b462de9248c5936842df896c60b0 (127.7.85.195:42885)
I20260812 00:10:55.252249 13263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0f2c81dde64f439b6002b84be66a41" candidate_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8564b462de9248c5936842df896c60b0"
I20260812 00:10:55.252254 12722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0f2c81dde64f439b6002b84be66a41" candidate_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d"
I20260812 00:10:55.252372 13263 raft_consensus.cc:3037] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:55.252367 12722 raft_consensus.cc:3037] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:55.253132 12722 raft_consensus.cc:2454] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85ac9013e1ca48a1bbaf26e75b23d956 in term 1.
I20260812 00:10:55.253275 13263 raft_consensus.cc:2454] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85ac9013e1ca48a1bbaf26e75b23d956 in term 1.
I20260812 00:10:55.253425 12851 leader_election.cc:302] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41afdfee8bbc4e8bbb1e22ebf3344e5d, 85ac9013e1ca48a1bbaf26e75b23d956; no voters:
I20260812 00:10:55.253553 13397 raft_consensus.cc:2781] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:55.253755 13397 raft_consensus.cc:686] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 1 LEADER]: Becoming Leader. State: Replica: 85ac9013e1ca48a1bbaf26e75b23d956, State: Running, Role: LEADER
I20260812 00:10:55.253875 13397 consensus_queue.cc:237] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } }
I20260812 00:10:55.254801 12416 catalog_manager.cc:5495] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85ac9013e1ca48a1bbaf26e75b23d956 (127.7.85.194). New cstate: current_term: 1 leader_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85ac9013e1ca48a1bbaf26e75b23d956" member_type: VOTER last_known_addr { host: "127.7.85.194" port: 35539 } health_report { overall_health: HEALTHY } } }
I20260812 00:10:55.272647 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.269990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.7.85.194:35539)
0812 00:10:55.270078 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:55.270080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:55.270090 (+ 10us) server_negotiation.cc:407] Connection header received
0812 00:10:55.270139 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:55.270139 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:55.270170 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:55.270245 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:10:55.270416 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.270905 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.271797 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:55.272004 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:55.272072 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:55.272272 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:10:55.272391 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:10:55.272461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:55.272540 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 00:10:55.272558 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:10:55.274893 13263 raft_consensus.cc:1261] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 1 FOLLOWER]: Refusing update from remote peer 85ac9013e1ca48a1bbaf26e75b23d956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:55.274892 12722 raft_consensus.cc:1261] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d [term 1 FOLLOWER]: Refusing update from remote peer 85ac9013e1ca48a1bbaf26e75b23d956: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:10:55.275290 13397 consensus_queue.cc:1035] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8564b462de9248c5936842df896c60b0" member_type: VOTER last_known_addr { host: "127.7.85.195" port: 42885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:55.275450 13403 consensus_queue.cc:1035] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41afdfee8bbc4e8bbb1e22ebf3344e5d" member_type: VOTER last_known_addr { host: "127.7.85.193" port: 38109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:10:55.278946 13412 mvcc.cc:205] Tried to move back new op lower bound from 7317477192802353152 to 7317477192720674816. Current Snapshot: MvccSnapshot[applied={T|T < 7317477192802353152 or (T in {7317477192802353152})}]
I20260812 00:10:55.281464 12416 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:55.281580 12416 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:10:55.282738 12416 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fffbd23667749e581542893b08672ce: Sending DeleteTablet for 3 replicas of tablet 0c0f2c81dde64f439b6002b84be66a41
I20260812 00:10:55.283171 12692 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0f2c81dde64f439b6002b84be66a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:55 UTC) from {username='slave'} at 127.0.0.1:37002
I20260812 00:10:55.283209 13215 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0f2c81dde64f439b6002b84be66a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:55 UTC) from {username='slave'} at 127.0.0.1:40070
I20260812 00:10:55.283349 12944 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0f2c81dde64f439b6002b84be66a41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:10:55 UTC) from {username='slave'} at 127.0.0.1:52812
I20260812 00:10:55.283576 13442 tablet_replica.cc:330] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: stopping tablet replica
I20260812 00:10:55.283576 13440 tablet_replica.cc:330] T 0c0f2c81dde64f439b6002b84be66a41 P 41afdfee8bbc4e8bbb1e22ebf3344e5d: stopping tablet replica
I20260812 00:10:55.283697 13442 raft_consensus.cc:2229] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:10:55.283804 13441 tablet_replica.cc:330] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: stopping tablet replica
I20260812 00:10:55.283850 13442 raft_consensus.cc:2258] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:55.283900 13441 raft_consensus.cc:2229] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:10:55.283948 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12538
I20260812 00:10:55.283998 13441 raft_consensus.cc:2258] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:10:55.284461 13441 ts_tablet_manager.cc:1905] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:55.286617 13441 ts_tablet_manager.cc:1918] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:55.286691 13441 log.cc:1196] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-2/wal/wals/0c0f2c81dde64f439b6002b84be66a41
I20260812 00:10:55.286917 13441 ts_tablet_manager.cc:1939] T 0c0f2c81dde64f439b6002b84be66a41 P 8564b462de9248c5936842df896c60b0: Deleting consensus metadata
I20260812 00:10:55.287343 12385 catalog_manager.cc:4841] TS 8564b462de9248c5936842df896c60b0 (127.7.85.195:42885): tablet 0c0f2c81dde64f439b6002b84be66a41 (table test-table [id=f27eb4bae1794408ac3d2cc27f5da656]) successfully deleted
I20260812 00:10:55.295228 13442 ts_tablet_manager.cc:1905] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:10:55.326869 13442 ts_tablet_manager.cc:1918] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:10:55.327019 13442 log.cc:1196] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Deleting WAL directory at /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786493185917310-7511-0/minicluster-data/ts-1/wal/wals/0c0f2c81dde64f439b6002b84be66a41
I20260812 00:10:55.327309 13442 ts_tablet_manager.cc:1939] T 0c0f2c81dde64f439b6002b84be66a41 P 85ac9013e1ca48a1bbaf26e75b23d956: Deleting consensus metadata
I20260812 00:10:55.327854 12385 catalog_manager.cc:4841] TS 85ac9013e1ca48a1bbaf26e75b23d956 (127.7.85.194:35539): tablet 0c0f2c81dde64f439b6002b84be66a41 (table test-table [id=f27eb4bae1794408ac3d2cc27f5da656]) successfully deleted
W20260812 00:10:55.333334 12380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.85.193:38109 (error 108)
I20260812 00:10:55.333578 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12825
I20260812 00:10:55.342113 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.333782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38109 (local address 127.0.0.1:37012)
0812 00:10:55.341937 (+ 8155us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.342046 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8106}
W20260812 00:10:55.347883 12380 catalog_manager.cc:4584] TS 41afdfee8bbc4e8bbb1e22ebf3344e5d (127.7.85.193:38109): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c0f2c81dde64f439b6002b84be66a41: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38109: connect: Connection refused (error 111)
I20260812 00:10:55.353368 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13083
I20260812 00:10:55.400617 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.400458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38109 (local address 127.0.0.1:37026)
0812 00:10:55.400522 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.400558 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 00:10:55.402146 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:55.402062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.193:38109 (local address 127.0.0.1:37034)
0812 00:10:55.402097 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 00:10:55.402120 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.193:38109: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 00:10:55.418831 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 12342
2026-08-12T00:10:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (960 ms)
[----------] 6 tests from EncryptionPolicyTest (10326 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:10:55 dist-test-slave-z8x9 krb5kdc[13454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:10:55 dist-test-slave-z8x9 krb5kdc[13454](info): set up 2 sockets
Aug 12 00:10:55 dist-test-slave-z8x9 krb5kdc[13454](info): commencing operation
krb5kdc: starting...
W20260812 00:10:56.795544 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:10:56 dist-test-slave-z8x9 krb5kdc[13454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:10:56.946626 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:57.035257 13483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:57.035552 13483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:57.035598 13483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:57.037261 13483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:57.037422 13483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:57.037451 13483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:57.037477 13483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:57.037672 13483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:57.039986 13483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43013
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:57.040307 13483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:57.040567 13483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:57.046202 13493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:57.046245 13496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:57.046491 13483 server_base.cc:1034] running on GCE node
W20260812 00:10:57.046211 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:57.046831 13483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:57.047209 13483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:57.048393 13483 hybrid_clock.cc:648] HybridClock initialized: now 1786493457048373 us; error 44 us; skew 500 ppm
Aug 12 00:10:57 dist-test-slave-z8x9 krb5kdc[13454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493457, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:57.051285 13483 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:10:57.051795 13483 webserver.cc:460] Webserver started at http://127.0.0.1:33289/ using document root <none> and password file <none>
I20260812 00:10:57.052045 13483 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:57.052095 13483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:57.052237 13483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:57.053515 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "7483de138f60496488f8ce7fb1d1713f"
format_stamp: "Formatted at 2026-08-12 00:10:57 on dist-test-slave-z8x9"
server_key: "08c1715ad8da987263a7db626565f086"
server_key_iv: "9c5494a74dd7e9081b2fa4d2147542d7"
server_key_version: "encryptionkey@0"
I20260812 00:10:57.053921 13483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "7483de138f60496488f8ce7fb1d1713f"
format_stamp: "Formatted at 2026-08-12 00:10:57 on dist-test-slave-z8x9"
server_key: "08c1715ad8da987263a7db626565f086"
server_key_iv: "9c5494a74dd7e9081b2fa4d2147542d7"
server_key_version: "encryptionkey@0"
I20260812 00:10:57.055613 13483 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:10:57.056710 13514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:57.057096 13483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:57.057271 13483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "7483de138f60496488f8ce7fb1d1713f"
format_stamp: "Formatted at 2026-08-12 00:10:57 on dist-test-slave-z8x9"
server_key: "08c1715ad8da987263a7db626565f086"
server_key_iv: "9c5494a74dd7e9081b2fa4d2147542d7"
server_key_version: "encryptionkey@0"
I20260812 00:10:57.057375 13483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:57.074604 13483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:57.077124 13483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:57.077433 13483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:57.081683 13483 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43013
I20260812 00:10:57.081724 13640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43013 every 8 connection(s)
I20260812 00:10:57.082111 13483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:57.083141 13642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:57.083562 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13483
I20260812 00:10:57.083702 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:57.083925 7511 external_mini_cluster.cc:1428] Setting key 22eb5b70f2f0b258498df1484f4fdaac
I20260812 00:10:57.086395 13642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f: Bootstrap starting.
I20260812 00:10:57.087127 13642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:57.087451 13642 log.cc:824] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:57.088702 13642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f: No bootstrap required, opened a new log
I20260812 00:10:57.090837 13642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7483de138f60496488f8ce7fb1d1713f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260812 00:10:57.091022 13642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:57.091059 13642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7483de138f60496488f8ce7fb1d1713f, State: Initialized, Role: FOLLOWER
I20260812 00:10:57.091207 13642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7483de138f60496488f8ce7fb1d1713f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260812 00:10:57.091291 13642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:57.091323 13642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:57.091362 13642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 0 FOLLOWER]: Advancing to term 1
Aug 12 00:10:57 dist-test-slave-z8x9 krb5kdc[13454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:10:57.092011 13642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7483de138f60496488f8ce7fb1d1713f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260812 00:10:57.092134 13642 leader_election.cc:302] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7483de138f60496488f8ce7fb1d1713f; no voters:
I20260812 00:10:57.092324 13642 leader_election.cc:288] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:57.092358 13651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:57.092495 13651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [term 1 LEADER]: Becoming Leader. State: Replica: 7483de138f60496488f8ce7fb1d1713f, State: Running, Role: LEADER
I20260812 00:10:57.092578 13651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7483de138f60496488f8ce7fb1d1713f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } }
I20260812 00:10:57.092641 13642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:57.092916 13651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7483de138f60496488f8ce7fb1d1713f. Latest consensus state: current_term: 1 leader_uuid: "7483de138f60496488f8ce7fb1d1713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7483de138f60496488f8ce7fb1d1713f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260812 00:10:57.092990 13651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:57.093246 13661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:57.093794 13661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:57.092972 13652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7483de138f60496488f8ce7fb1d1713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7483de138f60496488f8ce7fb1d1713f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43013 } } }
I20260812 00:10:57.094985 13652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:57.095968 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:57.084529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55383 (local address 127.0.0.1:43013)
0812 00:10:57.084724 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:57.084734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:57.084770 (+ 36us) server_negotiation.cc:407] Connection header received
0812 00:10:57.085357 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:57.085366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:57.085428 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:57.085583 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:57.086262 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:57.087636 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:57.088572 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:57.089207 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:57.089323 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:57.091926 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:57.091930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:57.091931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:57.091944 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:57.094232 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:57.094739 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:57.094742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:57.094743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:57.094860 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:57.095111 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:57.095112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:57.095112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:57.095377 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:57.095535 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:57.095677 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 00:10:57.095786 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260812 00:10:57.097237 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:10:57.098584 13661 catalog_manager.cc:1347] Generated new cluster ID: c2ff56675f714fda89bcf9917bdd463c
I20260812 00:10:57.098635 13661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:57.113056 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:57.101533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55091 (local address 127.0.0.1:43013)
0812 00:10:57.101581 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:57.101583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:57.105378 (+ 3795us) server_negotiation.cc:407] Connection header received
0812 00:10:57.105421 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:57.105423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:57.105437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:57.105513 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:57.106187 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:57.107018 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:57.108059 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:57.108402 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:57.108479 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:57.110041 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:57.110042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:57.110043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:10:57.110051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:57.111745 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:57.112166 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:57.112167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:57.112168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:57.112234 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:10:57.112469 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:10:57.112469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:10:57.112470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:10:57.112615 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:57.112727 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:57.112880 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:10:57.112953 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:10:57.115477 13661 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:57.116122 13661 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:57.118533 13661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7483de138f60496488f8ce7fb1d1713f: Generated new TSK 0
I20260812 00:10:57.118767 13661 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:10:57.122745 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13483
W20260812 00:10:57.159360 13448 connection.cc:535] client connection to 127.0.0.1:43013 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43013): Connection reset by peer (error 104)
2026-08-12T00:10:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1723 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:10:57 dist-test-slave-z8x9 krb5kdc[13697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:10:57 dist-test-slave-z8x9 krb5kdc[13697](info): set up 2 sockets
Aug 12 00:10:57 dist-test-slave-z8x9 krb5kdc[13697](info): commencing operation
krb5kdc: starting...
W20260812 00:10:58.499974 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:10:58 dist-test-slave-z8x9 krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:10:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:10:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:10:58.637048 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41439
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:10:58.728399 13728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:10:58.728652 13728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:10:58.728677 13728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:10:58.729974 13728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:10:58.730026 13728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:10:58.730039 13728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:10:58.730052 13728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:10:58.730065 13728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:10:58.731658 13728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41439
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:10:58.731948 13728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:10:58.732195 13728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:10:58.737591 13742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:58.737555 13749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:10:58.737624 13744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:10:58.737710 13728 server_base.cc:1034] running on GCE node
I20260812 00:10:58.738070 13728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:10:58.738427 13728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:10:58.739621 13728 hybrid_clock.cc:648] HybridClock initialized: now 1786493458739573 us; error 65 us; skew 500 ppm
Aug 12 00:10:58 dist-test-slave-z8x9 krb5kdc[13697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493458, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:10:58.747452 13728 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:10:58.748026 13728 webserver.cc:460] Webserver started at http://127.0.0.1:46145/ using document root <none> and password file <none>
I20260812 00:10:58.748319 13728 fs_manager.cc:359] Metadata directory not provided
I20260812 00:10:58.748397 13728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:10:58.748553 13728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:10:58.749740 13728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "10ca357982924280b1741ef1e7449918"
format_stamp: "Formatted at 2026-08-12 00:10:58 on dist-test-slave-z8x9"
server_key: "2d3d1688182315a640466b862c0d04f8"
server_key_iv: "4ff67b6a57321417d8cdb1f5e71fbc9a"
server_key_version: "encryptionkey@0"
I20260812 00:10:58.750110 13728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "10ca357982924280b1741ef1e7449918"
format_stamp: "Formatted at 2026-08-12 00:10:58 on dist-test-slave-z8x9"
server_key: "2d3d1688182315a640466b862c0d04f8"
server_key_iv: "4ff67b6a57321417d8cdb1f5e71fbc9a"
server_key_version: "encryptionkey@0"
I20260812 00:10:58.751564 13728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 00:10:58.752451 13759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:10:58.752719 13728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:10:58.752856 13728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "10ca357982924280b1741ef1e7449918"
format_stamp: "Formatted at 2026-08-12 00:10:58 on dist-test-slave-z8x9"
server_key: "2d3d1688182315a640466b862c0d04f8"
server_key_iv: "4ff67b6a57321417d8cdb1f5e71fbc9a"
server_key_version: "encryptionkey@0"
I20260812 00:10:58.752941 13728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:10:58.785375 13728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:10:58.788630 13728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:10:58.788877 13728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:10:58.793612 13728 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41439
I20260812 00:10:58.793672 13862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41439 every 8 connection(s)
I20260812 00:10:58.794075 13728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:10:58.795218 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:10:58.797914 13863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918: Bootstrap starting.
I20260812 00:10:58.798791 13863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918: Neither blocks nor log segments found. Creating new log.
I20260812 00:10:58.799044 13863 log.cc:824] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918: Log is configured to *not* fsync() on all Append() calls
I20260812 00:10:58.799959 13863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918: No bootstrap required, opened a new log
I20260812 00:10:58.801741 13863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10ca357982924280b1741ef1e7449918" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41439 } }
I20260812 00:10:58.801895 13863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:10:58.801918 13863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ca357982924280b1741ef1e7449918, State: Initialized, Role: FOLLOWER
I20260812 00:10:58.802088 13863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10ca357982924280b1741ef1e7449918" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41439 } }
I20260812 00:10:58.802155 13863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:10:58.802176 13863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:10:58.802205 13863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:10:58.802989 13863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10ca357982924280b1741ef1e7449918" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41439 } }
I20260812 00:10:58.803090 13863 leader_election.cc:302] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10ca357982924280b1741ef1e7449918; no voters:
I20260812 00:10:58.803329 13863 leader_election.cc:288] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:10:58.803654 13863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:10:58.803617 13869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:10:58.804133 13869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [term 1 LEADER]: Becoming Leader. State: Replica: 10ca357982924280b1741ef1e7449918, State: Running, Role: LEADER
I20260812 00:10:58.804353 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13728
I20260812 00:10:58.804337 13869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10ca357982924280b1741ef1e7449918" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41439 } }
I20260812 00:10:58.804479 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:10:58.804744 7511 external_mini_cluster.cc:1428] Setting key 07173ca232093f8c6a6c41ac06272ed2
I20260812 00:10:58.804765 13869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10ca357982924280b1741ef1e7449918. Latest consensus state: current_term: 1 leader_uuid: "10ca357982924280b1741ef1e7449918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10ca357982924280b1741ef1e7449918" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41439 } } }
I20260812 00:10:58.804836 13869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:58.805084 13873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10ca357982924280b1741ef1e7449918" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10ca357982924280b1741ef1e7449918" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41439 } } }
I20260812 00:10:58.805187 13873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918 [sys.catalog]: This master's current role is: LEADER
I20260812 00:10:58.805610 13890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:10:58.806246 13890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:10:58.808048 13890 catalog_manager.cc:1347] Generated new cluster ID: f0d6d844a6dc4449840a1d19aad5fa24
I20260812 00:10:58.808115 13890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:10:58.814018 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:58.805470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59907 (local address 127.0.0.1:41439)
0812 00:10:58.809409 (+ 3939us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:58.809417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:58.809451 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:10:58.809554 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:58.809560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:58.809605 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:58.809766 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:58.810016 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:58.810969 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:58.811753 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:58.812199 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:58.812332 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:58.813104 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:58.813108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:58.813108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:58.813375 (+ 267us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:58.813409 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:58.813421 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:58.813522 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:58.813604 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 00:10:58.813638 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3878,"thread_start_us":93,"threads_started":1}
I20260812 00:10:58.815040 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:10:58.819640 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:10:58.816543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60331 (local address 127.0.0.1:41439)
0812 00:10:58.816585 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:10:58.816586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:10:58.816711 (+ 125us) server_negotiation.cc:407] Connection header received
0812 00:10:58.816784 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:10:58.816785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:10:58.816796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:10:58.816844 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:10:58.816970 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:58.817583 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:58.818339 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:10:58.818557 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:10:58.818638 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:10:58.819380 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:10:58.819382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:10:58.819382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:10:58.819388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:10:58.819403 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:10:58.819407 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:10:58.819462 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:10:58.819523 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 00:10:58.819535 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:10:58.839673 13890 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:10:58.840471 13890 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:10:58.850046 13890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10ca357982924280b1741ef1e7449918: Generated new TSK 0
I20260812 00:10:58.850309 13890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:10:58.860239 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13728
W20260812 00:10:58.872251 13693 connection.cc:535] client connection to 127.0.0.1:41439 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41439): Connection reset by peer (error 104)
2026-08-12T00:10:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1713 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:10:58 dist-test-slave-z8x9 krb5kdc[13945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:10:58 dist-test-slave-z8x9 krb5kdc[13945](info): set up 2 sockets
Aug 12 00:10:58 dist-test-slave-z8x9 krb5kdc[13945](info): commencing operation
krb5kdc: starting...
W20260812 00:11:00.230608 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:00 dist-test-slave-z8x9 krb5kdc[13945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:00.373839 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:00.458619 13973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:00.458839 13973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:00.458863 13973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:00.460031 13973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:00.460080 13973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:00.460094 13973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:00.460107 13973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:00.460121 13973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:00.461710 13973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:00.462000 13973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:00.462253 13973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:00.468010 13995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:00.468039 13988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:00.468020 13991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:00.468052 13973 server_base.cc:1034] running on GCE node
I20260812 00:11:00.468557 13973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:00.468879 13973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:00.470049 13973 hybrid_clock.cc:648] HybridClock initialized: now 1786493460470030 us; error 44 us; skew 500 ppm
Aug 12 00:11:00 dist-test-slave-z8x9 krb5kdc[13945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:00.472699 13973 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:00.473272 13973 webserver.cc:460] Webserver started at http://127.0.0.1:36713/ using document root <none> and password file <none>
I20260812 00:11:00.473529 13973 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:00.473603 13973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:00.473757 13973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:00.474861 13973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "490117346e0840179ea443168e9e57f7"
format_stamp: "Formatted at 2026-08-12 00:11:00 on dist-test-slave-z8x9"
server_key: "390a4ebe7116c5508c8375e6f4e5bbd7"
server_key_iv: "f443482bc2e91169428e24f0e463ac96"
server_key_version: "encryptionkey@0"
I20260812 00:11:00.475220 13973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "490117346e0840179ea443168e9e57f7"
format_stamp: "Formatted at 2026-08-12 00:11:00 on dist-test-slave-z8x9"
server_key: "390a4ebe7116c5508c8375e6f4e5bbd7"
server_key_iv: "f443482bc2e91169428e24f0e463ac96"
server_key_version: "encryptionkey@0"
I20260812 00:11:00.476656 13973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:11:00.477900 14008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:00.478327 13973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:00.478464 13973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "490117346e0840179ea443168e9e57f7"
format_stamp: "Formatted at 2026-08-12 00:11:00 on dist-test-slave-z8x9"
server_key: "390a4ebe7116c5508c8375e6f4e5bbd7"
server_key_iv: "f443482bc2e91169428e24f0e463ac96"
server_key_version: "encryptionkey@0"
I20260812 00:11:00.478554 13973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:00.483176 13973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:00.485422 13973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:00.485589 13973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:00.490629 13973 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34261
I20260812 00:11:00.490664 14096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34261 every 8 connection(s)
I20260812 00:11:00.491057 13973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:00.492152 14098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:00.494630 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7: Bootstrap starting.
I20260812 00:11:00.495254 14098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:00.495584 14098 log.cc:824] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:00.496419 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7: No bootstrap required, opened a new log
I20260812 00:11:00.498513 14098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490117346e0840179ea443168e9e57f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34261 } }
I20260812 00:11:00.498673 14098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:00.498698 14098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490117346e0840179ea443168e9e57f7, State: Initialized, Role: FOLLOWER
I20260812 00:11:00.498812 14098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490117346e0840179ea443168e9e57f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34261 } }
I20260812 00:11:00.498942 14098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:00.498981 14098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:00.499012 14098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:00.499610 14098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490117346e0840179ea443168e9e57f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34261 } }
I20260812 00:11:00.499708 14098 leader_election.cc:302] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 490117346e0840179ea443168e9e57f7; no voters:
I20260812 00:11:00.499950 14098 leader_election.cc:288] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:00.500007 14105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:00.500285 14098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:00.500309 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 13973
I20260812 00:11:00.500442 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:00.500510 14105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [term 1 LEADER]: Becoming Leader. State: Replica: 490117346e0840179ea443168e9e57f7, State: Running, Role: LEADER
I20260812 00:11:00.500607 14105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490117346e0840179ea443168e9e57f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34261 } }
I20260812 00:11:00.500674 7511 external_mini_cluster.cc:1428] Setting key 132064945b3cef7aa6a95fccdecf91fd
I20260812 00:11:00.501430 14109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "490117346e0840179ea443168e9e57f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490117346e0840179ea443168e9e57f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34261 } } }
I20260812 00:11:00.501408 14111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 490117346e0840179ea443168e9e57f7. Latest consensus state: current_term: 1 leader_uuid: "490117346e0840179ea443168e9e57f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "490117346e0840179ea443168e9e57f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34261 } } }
I20260812 00:11:00.501547 14109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:00.501552 14111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:00.502154 14126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:00.503221 14126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:00.503638 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:00.501242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50165 (local address 127.0.0.1:34261)
0812 00:11:00.501963 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:00.501974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:00.502012 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:11:00.502150 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:00.502158 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:00.502269 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:00.502416 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:00.503156 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:00.503160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:00.503160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:00.503165 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:00.503220 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:00.503232 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:00.503319 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:00.503376 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 00:11:00.503420 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":80,"threads_started":1}
I20260812 00:11:00.504881 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:00.504946 14126 catalog_manager.cc:1347] Generated new cluster ID: 966887e5840c4c90ad97934cded0b477
I20260812 00:11:00.504982 14126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:00.519562 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:00.506485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56471 (local address 127.0.0.1:34261)
0812 00:11:00.506518 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:00.506519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:00.517501 (+ 10982us) server_negotiation.cc:407] Connection header received
0812 00:11:00.517762 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:00.517764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:00.517772 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:00.517831 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:00.519258 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:00.519260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:00.519260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:00.519266 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:00.519289 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:00.519295 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:00.519366 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:00.519443 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 00:11:00.519458 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 00:11:00.523303 14126 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:00.523517 14126 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:00.527534 14126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 490117346e0840179ea443168e9e57f7: Generated new TSK 0
I20260812 00:11:00.527882 14126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:00.530983 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 13973
2026-08-12T00:11:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1687 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:00 dist-test-slave-z8x9 krb5kdc[14164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:00 dist-test-slave-z8x9 krb5kdc[14164](info): set up 2 sockets
Aug 12 00:11:00 dist-test-slave-z8x9 krb5kdc[14164](info): commencing operation
krb5kdc: starting...
W20260812 00:11:01.894349 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:01 dist-test-slave-z8x9 krb5kdc[14164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:02.041652 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:02.127182 14192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:02.127420 14192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:02.127445 14192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:02.128546 14192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:02.128587 14192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:02.128602 14192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:02.128614 14192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:02.128628 14192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:02.130191 14192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:02.130407 14192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:02.130589 14192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:02.135887 14201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:02.135823 14208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:02.135814 14214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:02.136056 14192 server_base.cc:1034] running on GCE node
I20260812 00:11:02.136422 14192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:02.136708 14192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:02.137889 14192 hybrid_clock.cc:648] HybridClock initialized: now 1786493462137877 us; error 33 us; skew 500 ppm
Aug 12 00:11:02 dist-test-slave-z8x9 krb5kdc[14164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:02.141490 14192 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:02.142326 14192 webserver.cc:460] Webserver started at http://127.0.0.1:38021/ using document root <none> and password file <none>
I20260812 00:11:02.142571 14192 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:02.142618 14192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:02.142740 14192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:02.143846 14192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "02067d9db3fc42bb81a4a727b831c39b"
format_stamp: "Formatted at 2026-08-12 00:11:02 on dist-test-slave-z8x9"
server_key: "14e1cdf92698b661146b31136de0e11b"
server_key_iv: "6404268cc96a69655f11c8aaccbbfdd1"
server_key_version: "encryptionkey@0"
I20260812 00:11:02.144173 14192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "02067d9db3fc42bb81a4a727b831c39b"
format_stamp: "Formatted at 2026-08-12 00:11:02 on dist-test-slave-z8x9"
server_key: "14e1cdf92698b661146b31136de0e11b"
server_key_iv: "6404268cc96a69655f11c8aaccbbfdd1"
server_key_version: "encryptionkey@0"
I20260812 00:11:02.145668 14192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:11:02.146651 14224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:02.146955 14192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:02.147047 14192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "02067d9db3fc42bb81a4a727b831c39b"
format_stamp: "Formatted at 2026-08-12 00:11:02 on dist-test-slave-z8x9"
server_key: "14e1cdf92698b661146b31136de0e11b"
server_key_iv: "6404268cc96a69655f11c8aaccbbfdd1"
server_key_version: "encryptionkey@0"
I20260812 00:11:02.147152 14192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:02.159197 14192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:02.161791 14192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:02.161949 14192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:02.166630 14192 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40115
I20260812 00:11:02.166666 14320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40115 every 8 connection(s)
I20260812 00:11:02.167049 14192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:02.167850 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14192
I20260812 00:11:02.168002 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:02.168227 7511 external_mini_cluster.cc:1428] Setting key 3ecbe7d30cb29c4b3e411b3947cacb31
I20260812 00:11:02.168395 14321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:02.172403 14321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b: Bootstrap starting.
I20260812 00:11:02.173156 14321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:02.173571 14321 log.cc:824] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:02.174539 14321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b: No bootstrap required, opened a new log
Aug 12 00:11:02 dist-test-slave-z8x9 krb5kdc[14164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:11:02.176847 14321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02067d9db3fc42bb81a4a727b831c39b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260812 00:11:02.177019 14321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:02.177073 14321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02067d9db3fc42bb81a4a727b831c39b, State: Initialized, Role: FOLLOWER
I20260812 00:11:02.177284 14321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02067d9db3fc42bb81a4a727b831c39b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260812 00:11:02.177389 14321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:02.177452 14321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:02.177510 14321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:02.178092 14321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02067d9db3fc42bb81a4a727b831c39b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260812 00:11:02.178221 14321 leader_election.cc:302] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02067d9db3fc42bb81a4a727b831c39b; no voters:
I20260812 00:11:02.178422 14321 leader_election.cc:288] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:02.178468 14332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:02.178647 14332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [term 1 LEADER]: Becoming Leader. State: Replica: 02067d9db3fc42bb81a4a727b831c39b, State: Running, Role: LEADER
I20260812 00:11:02.178730 14321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:02.178808 14332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02067d9db3fc42bb81a4a727b831c39b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260812 00:11:02.179206 14334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02067d9db3fc42bb81a4a727b831c39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02067d9db3fc42bb81a4a727b831c39b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260812 00:11:02.179292 14334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:02.179522 14335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02067d9db3fc42bb81a4a727b831c39b. Latest consensus state: current_term: 1 leader_uuid: "02067d9db3fc42bb81a4a727b831c39b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02067d9db3fc42bb81a4a727b831c39b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260812 00:11:02.179896 14335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:02.180037 14349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:02.180621 14349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:02.180804 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:02.168860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60819 (local address 127.0.0.1:40115)
0812 00:11:02.169055 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:02.169063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:02.169089 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:11:02.169717 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:02.169724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:02.169773 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:02.169899 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:02.170912 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:02.171874 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:02.172998 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:02.173515 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:02.173600 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:02.176480 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:02.176485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:02.176486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:02.176498 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:02.178464 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:02.178916 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:02.178919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:02.178920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:02.179020 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:02.179295 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:02.179296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:02.179297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:02.179683 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:02.179849 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:02.180523 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 00:11:02.180656 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260812 00:11:02.182205 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:02.182700 14349 catalog_manager.cc:1347] Generated new cluster ID: f19e72a29f124119a1147d91e710331c
I20260812 00:11:02.182744 14349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:02.191403 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:02.183745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59229 (local address 127.0.0.1:40115)
0812 00:11:02.183786 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:02.183788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:02.183955 (+ 167us) server_negotiation.cc:407] Connection header received
0812 00:11:02.184018 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:02.184019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:02.184029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:02.184076 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:02.184822 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:02.185567 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:02.186588 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:02.186898 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:02.186957 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:02.188501 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:02.188502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:02.188503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:02.188510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:02.190177 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:02.190593 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:02.190594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:02.190594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:02.190662 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:02.190876 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:02.190877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:02.190877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:02.190997 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:02.191082 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:02.191235 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:11:02.191309 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 00:11:02.206784 14349 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:02.207576 14349 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:02.217667 14349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02067d9db3fc42bb81a4a727b831c39b: Generated new TSK 0
I20260812 00:11:02.217900 14349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:02.223250 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14192
2026-08-12T00:11:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1679 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:02 dist-test-slave-z8x9 krb5kdc[14382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:02 dist-test-slave-z8x9 krb5kdc[14382](info): set up 2 sockets
Aug 12 00:11:02 dist-test-slave-z8x9 krb5kdc[14382](info): commencing operation
krb5kdc: starting...
W20260812 00:11:03.597563 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:03 dist-test-slave-z8x9 krb5kdc[14382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:03.770239 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:03.867161 14412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:03.867427 14412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:03.867471 14412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:03.868788 14412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:03.868883 14412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:03.868914 14412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:03.868948 14412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:03.868973 14412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:03.870702 14412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:03.871025 14412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:03.871285 14412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:03.877218 14423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:03.877214 14428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:03.877511 14412 server_base.cc:1034] running on GCE node
W20260812 00:11:03.877214 14418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:03.877884 14412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:03.878278 14412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:03.879516 14412 hybrid_clock.cc:648] HybridClock initialized: now 1786493463879482 us; error 54 us; skew 500 ppm
Aug 12 00:11:03 dist-test-slave-z8x9 krb5kdc[14382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493463, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:03.887776 14412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:03.888366 14412 webserver.cc:460] Webserver started at http://127.0.0.1:36361/ using document root <none> and password file <none>
I20260812 00:11:03.888667 14412 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:03.888746 14412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:03.888911 14412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:03.890271 14412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "f6676fbf9f5f48bc8f79889abe29afc8"
format_stamp: "Formatted at 2026-08-12 00:11:03 on dist-test-slave-z8x9"
server_key: "1c39ff3ed24524d94907559162d7cf43"
server_key_iv: "0a45f922d7dcb549cbc8d658e08e3d36"
server_key_version: "encryptionkey@0"
I20260812 00:11:03.890673 14412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "f6676fbf9f5f48bc8f79889abe29afc8"
format_stamp: "Formatted at 2026-08-12 00:11:03 on dist-test-slave-z8x9"
server_key: "1c39ff3ed24524d94907559162d7cf43"
server_key_iv: "0a45f922d7dcb549cbc8d658e08e3d36"
server_key_version: "encryptionkey@0"
I20260812 00:11:03.892400 14412 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:11:03.893868 14439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:03.894229 14412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:11:03.894347 14412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "f6676fbf9f5f48bc8f79889abe29afc8"
format_stamp: "Formatted at 2026-08-12 00:11:03 on dist-test-slave-z8x9"
server_key: "1c39ff3ed24524d94907559162d7cf43"
server_key_iv: "0a45f922d7dcb549cbc8d658e08e3d36"
server_key_version: "encryptionkey@0"
I20260812 00:11:03.894483 14412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:03.924166 14412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:03.926923 14412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:03.927157 14412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:03.932885 14412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43779
I20260812 00:11:03.933524 14412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:03.932901 14535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43779 every 8 connection(s)
I20260812 00:11:03.935106 14536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:03.938755 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14412
I20260812 00:11:03.938898 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:03.939082 7511 external_mini_cluster.cc:1428] Setting key 3613d514f86f0ef3632d7fbb48fde569
I20260812 00:11:03.939620 14536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8: Bootstrap starting.
I20260812 00:11:03.940577 14536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:03.940954 14536 log.cc:824] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:03.942132 14536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8: No bootstrap required, opened a new log
I20260812 00:11:03.944671 14536 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260812 00:11:03.944849 14536 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:03.944917 14536 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6676fbf9f5f48bc8f79889abe29afc8, State: Initialized, Role: FOLLOWER
I20260812 00:11:03.945102 14536 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260812 00:11:03.945437 14536 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:03.945533 14536 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:03.945619 14536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:03.946386 14536 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260812 00:11:03.946532 14536 leader_election.cc:302] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6676fbf9f5f48bc8f79889abe29afc8; no voters:
I20260812 00:11:03.946847 14545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:03.946820 14536 leader_election.cc:288] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:03.947062 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:03.939831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52171 (local address 127.0.0.1:43779)
0812 00:11:03.940072 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:03.940085 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:03.940300 (+ 215us) server_negotiation.cc:407] Connection header received
0812 00:11:03.941039 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:03.941048 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:03.941117 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:03.941300 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:03.941570 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:03.943070 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:03.944260 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:03.945040 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:03.945285 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:03.946067 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:03.946072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:03.946072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:03.946300 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:03.946346 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:03.946363 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:03.946435 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:03.946609 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 00:11:03.946694 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":116,"threads_started":1}
I20260812 00:11:03.947103 14545 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [term 1 LEADER]: Becoming Leader. State: Replica: f6676fbf9f5f48bc8f79889abe29afc8, State: Running, Role: LEADER
I20260812 00:11:03.947424 14536 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:03.947762 14545 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } }
I20260812 00:11:03.948269 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:03.948256 14548 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } } }
I20260812 00:11:03.948342 14548 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:03.948432 14547 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6676fbf9f5f48bc8f79889abe29afc8. Latest consensus state: current_term: 1 leader_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6676fbf9f5f48bc8f79889abe29afc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43779 } } }
I20260812 00:11:03.948488 14547 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:03.948611 14553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:03.949141 14553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:03.954900 14553 catalog_manager.cc:1347] Generated new cluster ID: c0d81109e65546aeb8baa59bf4e52f1a
I20260812 00:11:03.954989 14553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:03.958922 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:03.953753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36987 (local address 127.0.0.1:43779)
0812 00:11:03.953801 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:03.953805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:03.954242 (+ 437us) server_negotiation.cc:407] Connection header received
0812 00:11:03.954557 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:03.954558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:03.954571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:03.954604 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:03.954871 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:03.955608 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:03.957006 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:03.957455 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:03.957544 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:03.958558 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:03.958559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:03.958560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:03.958567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:03.958586 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:03.958592 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:03.958624 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:03.958750 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 00:11:03.958781 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:11:03.971282 14553 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:03.972119 14553 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:03.980615 14553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6676fbf9f5f48bc8f79889abe29afc8: Generated new TSK 0
I20260812 00:11:03.980868 14553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:03.989318 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14412
2026-08-12T00:11:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1774 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:04 dist-test-slave-z8x9 krb5kdc[14599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:04 dist-test-slave-z8x9 krb5kdc[14599](info): set up 2 sockets
Aug 12 00:11:04 dist-test-slave-z8x9 krb5kdc[14599](info): commencing operation
krb5kdc: starting...
W20260812 00:11:05.391079 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:05 dist-test-slave-z8x9 krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:05.549237 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:05.641999 14622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:05.642293 14622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:05.642335 14622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:05.643548 14622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:05.643632 14622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:05.643659 14622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:05.643692 14622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:05.643718 14622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:05.645442 14622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:05.645762 14622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:05.646034 14622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:05.651403 14631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:05.651448 14629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:05.651407 14642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:05.652068 14622 server_base.cc:1034] running on GCE node
I20260812 00:11:05.652376 14622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:05.652767 14622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:05.653921 14622 hybrid_clock.cc:648] HybridClock initialized: now 1786493465653894 us; error 43 us; skew 500 ppm
Aug 12 00:11:05 dist-test-slave-z8x9 krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:05.657464 14622 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:05.657999 14622 webserver.cc:460] Webserver started at http://127.0.0.1:34517/ using document root <none> and password file <none>
I20260812 00:11:05.658255 14622 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:05.658308 14622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:05.658452 14622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:05.659595 14622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "47bca2c4a16a4f41aba3676ce917be28"
format_stamp: "Formatted at 2026-08-12 00:11:05 on dist-test-slave-z8x9"
server_key: "c21fa475530415e8b47200952505f089"
server_key_iv: "fc22da0ebb8986c8ebf1083e31b7b21b"
server_key_version: "encryptionkey@0"
I20260812 00:11:05.660089 14622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "47bca2c4a16a4f41aba3676ce917be28"
format_stamp: "Formatted at 2026-08-12 00:11:05 on dist-test-slave-z8x9"
server_key: "c21fa475530415e8b47200952505f089"
server_key_iv: "fc22da0ebb8986c8ebf1083e31b7b21b"
server_key_version: "encryptionkey@0"
I20260812 00:11:05.661598 14622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:11:05.662542 14652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:05.662869 14622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 00:11:05.662961 14622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "47bca2c4a16a4f41aba3676ce917be28"
format_stamp: "Formatted at 2026-08-12 00:11:05 on dist-test-slave-z8x9"
server_key: "c21fa475530415e8b47200952505f089"
server_key_iv: "fc22da0ebb8986c8ebf1083e31b7b21b"
server_key_version: "encryptionkey@0"
I20260812 00:11:05.663082 14622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:05.667043 14622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:05.670457 14622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:05.670629 14622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:05.675098 14622 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42889
I20260812 00:11:05.675254 14754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42889 every 8 connection(s)
I20260812 00:11:05.675549 14622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:05.676666 14755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:05.676950 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14622
I20260812 00:11:05.677265 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:05.677553 7511 external_mini_cluster.cc:1428] Setting key e8358e5f792e3fc29e582abf0f2fdaa3
I20260812 00:11:05.680176 14755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28: Bootstrap starting.
I20260812 00:11:05.680668 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:05.678354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35231 (local address 127.0.0.1:42889)
0812 00:11:05.678555 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:05.678563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:05.678591 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:11:05.679027 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:05.679034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:05.679060 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:05.679181 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:05.679952 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:05.679955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:05.679955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:05.679960 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:05.680002 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:05.680013 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:05.680078 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:05.680435 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 00:11:05.680486 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260812 00:11:05.681015 14755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:05.681402 14755 log.cc:824] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:05.682358 14755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28: No bootstrap required, opened a new log
I20260812 00:11:05.682090 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:05.684441 14755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bca2c4a16a4f41aba3676ce917be28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260812 00:11:05.684669 14755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:05.684747 14755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bca2c4a16a4f41aba3676ce917be28, State: Initialized, Role: FOLLOWER
I20260812 00:11:05.684907 14755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bca2c4a16a4f41aba3676ce917be28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260812 00:11:05.685021 14755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:05.685109 14755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:05.685201 14755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:05.685523 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:05.684378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45851 (local address 127.0.0.1:42889)
0812 00:11:05.684427 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:05.684429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:05.684500 (+ 71us) server_negotiation.cc:407] Connection header received
0812 00:11:05.684561 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:05.684562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:05.684567 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:05.684629 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:05.685302 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:05.685304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:05.685304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:05.685309 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:05.685327 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:05.685332 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:05.685398 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:05.685449 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 00:11:05.685462 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 00:11:05.686095 14755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bca2c4a16a4f41aba3676ce917be28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260812 00:11:05.686196 14755 leader_election.cc:302] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bca2c4a16a4f41aba3676ce917be28; no voters:
I20260812 00:11:05.686362 14755 leader_election.cc:288] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:05.686491 14776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:05.686671 14755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:05.686698 14776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [term 1 LEADER]: Becoming Leader. State: Replica: 47bca2c4a16a4f41aba3676ce917be28, State: Running, Role: LEADER
I20260812 00:11:05.686911 14776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bca2c4a16a4f41aba3676ce917be28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } }
I20260812 00:11:05.687371 14781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bca2c4a16a4f41aba3676ce917be28. Latest consensus state: current_term: 1 leader_uuid: "47bca2c4a16a4f41aba3676ce917be28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bca2c4a16a4f41aba3676ce917be28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } } }
I20260812 00:11:05.687408 14777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47bca2c4a16a4f41aba3676ce917be28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bca2c4a16a4f41aba3676ce917be28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42889 } } }
I20260812 00:11:05.687479 14781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:05.687747 14777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:05.687776 14801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:05.688390 14801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:05.690335 14801 catalog_manager.cc:1347] Generated new cluster ID: fee7ff1157074d7b9c4eebdc1e024b63
I20260812 00:11:05.690400 14801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:05.708662 14801 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:05.708879 14801 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:05.719872 14801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47bca2c4a16a4f41aba3676ce917be28: Generated new TSK 0
I20260812 00:11:05.720114 14801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:05.728540 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14622
2026-08-12T00:11:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1728 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:05 dist-test-slave-z8x9 krb5kdc[14826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:05 dist-test-slave-z8x9 krb5kdc[14826](info): set up 2 sockets
Aug 12 00:11:05 dist-test-slave-z8x9 krb5kdc[14826](info): commencing operation
krb5kdc: starting...
W20260812 00:11:07.086035 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:07 dist-test-slave-z8x9 krb5kdc[14826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:07.237296 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:07.328260 14863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:07.328513 14863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:07.328539 14863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:07.329726 14863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:07.329767 14863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:07.329782 14863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:07.329794 14863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:07.329806 14863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:07.331295 14863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:07.331517 14863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:07.331723 14863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:07.338150 14880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:07.338337 14875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:07.338232 14884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:07.339116 14863 server_base.cc:1034] running on GCE node
I20260812 00:11:07.339457 14863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:07.339848 14863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:07.341075 14863 hybrid_clock.cc:648] HybridClock initialized: now 1786493467341053 us; error 34 us; skew 500 ppm
Aug 12 00:11:07 dist-test-slave-z8x9 krb5kdc[14826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493467, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:07.344883 14863 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:07.345485 14863 webserver.cc:460] Webserver started at http://127.0.0.1:35659/ using document root <none> and password file <none>
I20260812 00:11:07.345880 14863 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:07.345948 14863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:07.346100 14863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:07.347427 14863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "724aa89ffa864909bbc1d78135a7c5a9"
format_stamp: "Formatted at 2026-08-12 00:11:07 on dist-test-slave-z8x9"
server_key: "ede90f8a97fde510ea319bb000eb73de"
server_key_iv: "ae9f08c6903250611a7a9b62907b446c"
server_key_version: "encryptionkey@0"
I20260812 00:11:07.347924 14863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "724aa89ffa864909bbc1d78135a7c5a9"
format_stamp: "Formatted at 2026-08-12 00:11:07 on dist-test-slave-z8x9"
server_key: "ede90f8a97fde510ea319bb000eb73de"
server_key_iv: "ae9f08c6903250611a7a9b62907b446c"
server_key_version: "encryptionkey@0"
I20260812 00:11:07.349588 14863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 00:11:07.350780 14904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:07.351039 14863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:07.351148 14863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "724aa89ffa864909bbc1d78135a7c5a9"
format_stamp: "Formatted at 2026-08-12 00:11:07 on dist-test-slave-z8x9"
server_key: "ede90f8a97fde510ea319bb000eb73de"
server_key_iv: "ae9f08c6903250611a7a9b62907b446c"
server_key_version: "encryptionkey@0"
I20260812 00:11:07.351217 14863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:07.369545 14863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:07.372123 14863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:07.372262 14863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:07.376860 14863 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40077
I20260812 00:11:07.376895 15009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40077 every 8 connection(s)
I20260812 00:11:07.377393 14863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:07.378577 15012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:07.381111 15012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9: Bootstrap starting.
I20260812 00:11:07.381814 15012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:07.382081 15012 log.cc:824] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:07.382908 15012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9: No bootstrap required, opened a new log
I20260812 00:11:07.384595 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 14863
I20260812 00:11:07.384616 15012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724aa89ffa864909bbc1d78135a7c5a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40077 } }
I20260812 00:11:07.384819 15012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:07.384862 15012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724aa89ffa864909bbc1d78135a7c5a9, State: Initialized, Role: FOLLOWER
I20260812 00:11:07.384871 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:07.385037 15012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724aa89ffa864909bbc1d78135a7c5a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40077 } }
I20260812 00:11:07.385102 7511 external_mini_cluster.cc:1428] Setting key c7c325a0bdd7cf3ac01bb19a2ac159f4
I20260812 00:11:07.385130 15012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:07.385154 15012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:07.385309 15012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:07.386157 15012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724aa89ffa864909bbc1d78135a7c5a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40077 } }
I20260812 00:11:07.386307 15012 leader_election.cc:302] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 724aa89ffa864909bbc1d78135a7c5a9; no voters:
I20260812 00:11:07.386495 15012 leader_election.cc:288] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:07.386621 15025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:07.386763 15012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:07.386859 15025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [term 1 LEADER]: Becoming Leader. State: Replica: 724aa89ffa864909bbc1d78135a7c5a9, State: Running, Role: LEADER
I20260812 00:11:07.387032 15025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724aa89ffa864909bbc1d78135a7c5a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40077 } }
I20260812 00:11:07.387724 15028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "724aa89ffa864909bbc1d78135a7c5a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724aa89ffa864909bbc1d78135a7c5a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40077 } } }
I20260812 00:11:07.387871 15028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:07.387705 15025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 724aa89ffa864909bbc1d78135a7c5a9. Latest consensus state: current_term: 1 leader_uuid: "724aa89ffa864909bbc1d78135a7c5a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "724aa89ffa864909bbc1d78135a7c5a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40077 } } }
I20260812 00:11:07.387933 15025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9 [sys.catalog]: This master's current role is: LEADER
W20260812 00:11:07.388854 15056 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:07.388919 15056 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:07.388968 15052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:07.389153 15052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:07.390956 15052 catalog_manager.cc:1347] Generated new cluster ID: aafe338013c244c2aa635f965f84f0f1
I20260812 00:11:07.391021 15052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:11:07 dist-test-slave-z8x9 krb5kdc[14826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:11:07.399694 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:07.385756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:41907 (local address 127.0.0.1:40077)
0812 00:11:07.386311 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:07.386322 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:07.386359 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:11:07.386488 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:07.386495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:07.386562 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:07.386726 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:07.387761 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:07.389542 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:07.390646 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:07.391278 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:07.391388 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:07.394773 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:07.394779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:07.394780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:07.394796 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:07.397062 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:07.397641 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:07.397646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:07.397646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:07.397927 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:07.398212 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:07.398213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:07.398213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:07.398706 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:07.398930 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:07.399309 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 00:11:07.399437 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":435,"threads_started":1}
I20260812 00:11:07.401826 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:07.411547 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:07.403564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:40737 (local address 127.0.0.1:40077)
0812 00:11:07.403604 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:07.403606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:07.403617 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:11:07.403645 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:07.403646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:07.403657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:07.403721 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:07.404434 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:07.405276 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:07.406329 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:07.406662 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:07.406746 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:07.408288 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:07.408289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:07.408290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:07.408298 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:07.410057 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:07.410511 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:07.410513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:07.410513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:07.410586 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:07.410835 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:07.410836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:07.410836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:07.410984 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:07.411113 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:07.411245 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 00:11:07.411437 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:11:07.421365 15052 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:07.422128 15052 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:07.429010 15052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 724aa89ffa864909bbc1d78135a7c5a9: Generated new TSK 0
I20260812 00:11:07.429327 15052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:07.434687 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 14863
W20260812 00:11:07.453423 14823 connection.cc:535] client connection to 127.0.0.1:40077 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40077): Connection reset by peer (error 104)
2026-08-12T00:11:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1709 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:07 dist-test-slave-z8x9 krb5kdc[15081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:07 dist-test-slave-z8x9 krb5kdc[15081](info): set up 2 sockets
Aug 12 00:11:07 dist-test-slave-z8x9 krb5kdc[15081](info): commencing operation
krb5kdc: starting...
W20260812 00:11:08.818251 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:08 dist-test-slave-z8x9 krb5kdc[15081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:08.969290 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:09.066171 15103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:09.066414 15103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:09.066440 15103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:09.067719 15103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:09.067773 15103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:09.067788 15103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:09.067802 15103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:09.067816 15103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:09.069451 15103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:09.069751 15103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:09.070006 15103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:11:09.075441 15103 server_base.cc:1034] running on GCE node
W20260812 00:11:09.075423 15115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:09.075414 15117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:09.075423 15114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:09.076098 15103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:09.076452 15103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:09.077651 15103 hybrid_clock.cc:648] HybridClock initialized: now 1786493469077646 us; error 38 us; skew 500 ppm
Aug 12 00:11:09 dist-test-slave-z8x9 krb5kdc[15081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493469, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:09.086385 15103 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:09.086973 15103 webserver.cc:460] Webserver started at http://127.0.0.1:39393/ using document root <none> and password file <none>
I20260812 00:11:09.087229 15103 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:09.087301 15103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:09.087458 15103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:09.088608 15103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "1d94f71186144acbae35027f50974271"
format_stamp: "Formatted at 2026-08-12 00:11:09 on dist-test-slave-z8x9"
server_key: "2b375624dfbcff3954910709dc59e6dc"
server_key_iv: "f4aa19691c3523aae4d5b91b714c1730"
server_key_version: "encryptionkey@0"
I20260812 00:11:09.089023 15103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "1d94f71186144acbae35027f50974271"
format_stamp: "Formatted at 2026-08-12 00:11:09 on dist-test-slave-z8x9"
server_key: "2b375624dfbcff3954910709dc59e6dc"
server_key_iv: "f4aa19691c3523aae4d5b91b714c1730"
server_key_version: "encryptionkey@0"
I20260812 00:11:09.090572 15103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:09.091547 15136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:09.091883 15103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:09.092028 15103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "1d94f71186144acbae35027f50974271"
format_stamp: "Formatted at 2026-08-12 00:11:09 on dist-test-slave-z8x9"
server_key: "2b375624dfbcff3954910709dc59e6dc"
server_key_iv: "f4aa19691c3523aae4d5b91b714c1730"
server_key_version: "encryptionkey@0"
I20260812 00:11:09.092181 15103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:09.101644 15103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:09.103873 15103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:09.104049 15103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:09.108975 15103 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33779
I20260812 00:11:09.109009 15239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33779 every 8 connection(s)
I20260812 00:11:09.109449 15103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:09.110563 15242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:09.112874 15242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271: Bootstrap starting.
I20260812 00:11:09.113462 15242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:09.113713 15242 log.cc:824] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:09.114496 15242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271: No bootstrap required, opened a new log
I20260812 00:11:09.116072 15242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d94f71186144acbae35027f50974271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260812 00:11:09.116219 15242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:09.116242 15242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d94f71186144acbae35027f50974271, State: Initialized, Role: FOLLOWER
I20260812 00:11:09.116407 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15103
I20260812 00:11:09.116421 15242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d94f71186144acbae35027f50974271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260812 00:11:09.116525 15242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:09.116567 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:09.116568 15242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:09.116628 15242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:09.116797 7511 external_mini_cluster.cc:1428] Setting key 011d7c0ef596d5137ebb2d23f673ccf6
I20260812 00:11:09.117543 15242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d94f71186144acbae35027f50974271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260812 00:11:09.117666 15242 leader_election.cc:302] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d94f71186144acbae35027f50974271; no voters:
I20260812 00:11:09.117831 15242 leader_election.cc:288] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:09.117883 15263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:09.118053 15263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [term 1 LEADER]: Becoming Leader. State: Replica: 1d94f71186144acbae35027f50974271, State: Running, Role: LEADER
I20260812 00:11:09.118108 15242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:09.118268 15263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d94f71186144acbae35027f50974271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } }
I20260812 00:11:09.119694 15264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d94f71186144acbae35027f50974271. Latest consensus state: current_term: 1 leader_uuid: "1d94f71186144acbae35027f50974271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d94f71186144acbae35027f50974271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } } }
I20260812 00:11:09.119793 15264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:09.119764 15265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d94f71186144acbae35027f50974271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d94f71186144acbae35027f50974271" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33779 } } }
I20260812 00:11:09.119859 15265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:09.120002 15291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:09.121790 15291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:09.123558 15291 catalog_manager.cc:1347] Generated new cluster ID: 0d0349d603474d68b09139d84c86be00
I20260812 00:11:09.123615 15291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:09.123993 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:09.117394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:58383 (local address 127.0.0.1:33779)
0812 00:11:09.117630 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:09.117641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:09.117680 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:11:09.117823 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:09.117831 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:09.117898 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:09.118053 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:09.118307 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:09.119985 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:09.121296 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:09.121939 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:09.122055 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:09.123052 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:09.123055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:09.123056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:09.123293 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:09.123338 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:09.123354 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:09.123446 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:09.123727 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 00:11:09.123768 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
I20260812 00:11:09.125059 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:09.131688 15291 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:09.132220 15291 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:09.133065 15261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:09.126595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:53047 (local address 127.0.0.1:33779)
0812 00:11:09.126631 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:09.126632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:09.129364 (+ 2732us) server_negotiation.cc:407] Connection header received
0812 00:11:09.129401 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:09.129402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:09.129416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:09.129488 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:09.129712 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:09.130467 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:09.131541 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:09.131858 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:09.131901 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:09.132778 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:09.132780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:09.132781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:09.132786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:09.132801 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:09.132807 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:09.132840 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:09.132974 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 00:11:09.132987 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:11:09.143680 15291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d94f71186144acbae35027f50974271: Generated new TSK 0
I20260812 00:11:09.143900 15291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:09.149497 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15103
W20260812 00:11:09.166258 15075 connection.cc:535] client connection to 127.0.0.1:33779 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33779): Connection reset by peer (error 104)
2026-08-12T00:11:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1713 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:09 dist-test-slave-z8x9 krb5kdc[15312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:09 dist-test-slave-z8x9 krb5kdc[15312](info): set up 2 sockets
Aug 12 00:11:09 dist-test-slave-z8x9 krb5kdc[15312](info): commencing operation
krb5kdc: starting...
W20260812 00:11:10.519136 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:10 dist-test-slave-z8x9 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:10.673846 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:10.767381 15333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:10.767606 15333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:10.767630 15333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:10.768744 15333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:10.768786 15333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:10.768800 15333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:10.768812 15333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:10.768826 15333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:10.770434 15333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:10.770650 15333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:10.770838 15333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:10.776113 15344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:10.776419 15333 server_base.cc:1034] running on GCE node
W20260812 00:11:10.776484 15342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:10.776441 15340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:10.776975 15333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:10.777366 15333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:10.778578 15333 hybrid_clock.cc:648] HybridClock initialized: now 1786493470778568 us; error 39 us; skew 500 ppm
Aug 12 00:11:10 dist-test-slave-z8x9 krb5kdc[15312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:10.781896 15333 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:10.782480 15333 webserver.cc:460] Webserver started at http://127.0.0.1:39467/ using document root <none> and password file <none>
I20260812 00:11:10.782761 15333 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:10.782836 15333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:10.782999 15333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:10.784122 15333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "756b421329ba43a9b42cb530328b57bf"
format_stamp: "Formatted at 2026-08-12 00:11:10 on dist-test-slave-z8x9"
server_key: "632086e0cc7f707d7035f5b29f6e9a03"
server_key_iv: "dcb42f0b7f04a7f10122ccf6496af83d"
server_key_version: "encryptionkey@0"
I20260812 00:11:10.784513 15333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "756b421329ba43a9b42cb530328b57bf"
format_stamp: "Formatted at 2026-08-12 00:11:10 on dist-test-slave-z8x9"
server_key: "632086e0cc7f707d7035f5b29f6e9a03"
server_key_iv: "dcb42f0b7f04a7f10122ccf6496af83d"
server_key_version: "encryptionkey@0"
I20260812 00:11:10.786267 15333 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 00:11:10.787142 15356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:10.787389 15333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:10.787513 15333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "756b421329ba43a9b42cb530328b57bf"
format_stamp: "Formatted at 2026-08-12 00:11:10 on dist-test-slave-z8x9"
server_key: "632086e0cc7f707d7035f5b29f6e9a03"
server_key_iv: "dcb42f0b7f04a7f10122ccf6496af83d"
server_key_version: "encryptionkey@0"
I20260812 00:11:10.787595 15333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:10.791262 15333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:10.793649 15333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:10.793818 15333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:10.798697 15333 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40575
I20260812 00:11:10.798756 15457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40575 every 8 connection(s)
I20260812 00:11:10.799155 15333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:10.799995 15459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:10.800809 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15333
I20260812 00:11:10.800952 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:10.801220 7511 external_mini_cluster.cc:1428] Setting key 490aaccae6555a575a1fdf98b544b029
I20260812 00:11:10.803540 15459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf: Bootstrap starting.
I20260812 00:11:10.804168 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:10.801808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:47263 (local address 127.0.0.1:40575)
0812 00:11:10.801995 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:10.802003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:10.802029 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:11:10.802468 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:10.802474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:10.802499 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:10.802616 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:10.803456 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:10.803464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:10.803464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:10.803472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:10.803556 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:10.803573 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:10.803808 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:10.803900 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 00:11:10.803959 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260812 00:11:10.804376 15459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:10.804746 15459 log.cc:824] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:10.805274 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:10.805676 15459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf: No bootstrap required, opened a new log
I20260812 00:11:10.808209 15459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756b421329ba43a9b42cb530328b57bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260812 00:11:10.808377 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:10.806776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:58899 (local address 127.0.0.1:40575)
0812 00:11:10.806821 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:10.806823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:10.807039 (+ 216us) server_negotiation.cc:407] Connection header received
0812 00:11:10.807082 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:10.807083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:10.807120 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:10.807305 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:10.807996 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:10.807997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:10.807997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:10.808007 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:10.808080 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:10.808092 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:10.808160 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:10.808244 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 00:11:10.808265 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:11:10.808444 15459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:10.808497 15459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 756b421329ba43a9b42cb530328b57bf, State: Initialized, Role: FOLLOWER
I20260812 00:11:10.808617 15459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756b421329ba43a9b42cb530328b57bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260812 00:11:10.808717 15459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:10.808751 15459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:10.808790 15459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:10.809526 15459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756b421329ba43a9b42cb530328b57bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260812 00:11:10.809624 15459 leader_election.cc:302] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 756b421329ba43a9b42cb530328b57bf; no voters:
I20260812 00:11:10.809814 15459 leader_election.cc:288] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:10.809930 15477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:10.810096 15477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [term 1 LEADER]: Becoming Leader. State: Replica: 756b421329ba43a9b42cb530328b57bf, State: Running, Role: LEADER
I20260812 00:11:10.810175 15459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:10.810201 15477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756b421329ba43a9b42cb530328b57bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } }
I20260812 00:11:10.810545 15481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "756b421329ba43a9b42cb530328b57bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756b421329ba43a9b42cb530328b57bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } } }
I20260812 00:11:10.810555 15482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 756b421329ba43a9b42cb530328b57bf. Latest consensus state: current_term: 1 leader_uuid: "756b421329ba43a9b42cb530328b57bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "756b421329ba43a9b42cb530328b57bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40575 } } }
I20260812 00:11:10.810702 15481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:10.810734 15482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:10.810910 15489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:10.811473 15489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:10.813189 15489 catalog_manager.cc:1347] Generated new cluster ID: 3a0591ac9f604a20abff75758051db4a
I20260812 00:11:10.813256 15489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:10.833865 15489 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:10.834088 15489 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:10.839937 15489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 756b421329ba43a9b42cb530328b57bf: Generated new TSK 0
I20260812 00:11:10.840173 15489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:10.849761 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15333
2026-08-12T00:11:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1694 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:10 dist-test-slave-z8x9 krb5kdc[15529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:10 dist-test-slave-z8x9 krb5kdc[15529](info): set up 2 sockets
Aug 12 00:11:10 dist-test-slave-z8x9 krb5kdc[15529](info): commencing operation
krb5kdc: starting...
W20260812 00:11:12.211663 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:12 dist-test-slave-z8x9 krb5kdc[15529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:12.361575 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:12.453143 15551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:12.453404 15551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:12.453429 15551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:12.454524 15551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:12.454560 15551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:12.454574 15551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:12.454586 15551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:12.454599 15551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:12.456027 15551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33461
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:12.456233 15551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:12.456408 15551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:12.461882 15569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:12.461938 15568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:12.461882 15572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:12.462996 15551 server_base.cc:1034] running on GCE node
I20260812 00:11:12.463521 15551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:12.464051 15551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:12.465282 15551 hybrid_clock.cc:648] HybridClock initialized: now 1786493472465245 us; error 46 us; skew 500 ppm
Aug 12 00:11:12 dist-test-slave-z8x9 krb5kdc[15529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493472, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:12.468338 15551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:12.468917 15551 webserver.cc:460] Webserver started at http://127.0.0.1:37857/ using document root <none> and password file <none>
I20260812 00:11:12.469234 15551 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:12.469313 15551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:12.469484 15551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:12.470609 15551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "80d383f419b6465e97d26615e03be9fa"
format_stamp: "Formatted at 2026-08-12 00:11:12 on dist-test-slave-z8x9"
server_key: "770518ff8ddfa4c90d8dfaf67ba14323"
server_key_iv: "8b7eb15c448acce3c2bc463629dbb15b"
server_key_version: "encryptionkey@0"
I20260812 00:11:12.470997 15551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "80d383f419b6465e97d26615e03be9fa"
format_stamp: "Formatted at 2026-08-12 00:11:12 on dist-test-slave-z8x9"
server_key: "770518ff8ddfa4c90d8dfaf67ba14323"
server_key_iv: "8b7eb15c448acce3c2bc463629dbb15b"
server_key_version: "encryptionkey@0"
I20260812 00:11:12.472467 15551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:11:12.473451 15583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:12.473727 15551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:12.473855 15551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "80d383f419b6465e97d26615e03be9fa"
format_stamp: "Formatted at 2026-08-12 00:11:12 on dist-test-slave-z8x9"
server_key: "770518ff8ddfa4c90d8dfaf67ba14323"
server_key_iv: "8b7eb15c448acce3c2bc463629dbb15b"
server_key_version: "encryptionkey@0"
I20260812 00:11:12.473941 15551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:12.493526 15551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:12.496197 15551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:12.496428 15551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:12.500988 15551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33461
I20260812 00:11:12.501029 15667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33461 every 8 connection(s)
I20260812 00:11:12.501448 15551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:12.502574 15672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:12.505121 15672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa: Bootstrap starting.
I20260812 00:11:12.505875 15672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:12.506212 15672 log.cc:824] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:12.507069 15672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa: No bootstrap required, opened a new log
I20260812 00:11:12.508746 15672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d383f419b6465e97d26615e03be9fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260812 00:11:12.508895 15672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:12.508919 15672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d383f419b6465e97d26615e03be9fa, State: Initialized, Role: FOLLOWER
I20260812 00:11:12.509109 15672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d383f419b6465e97d26615e03be9fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260812 00:11:12.509239 15672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:12.509281 15672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:12.509341 15672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:12.509328 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15551
I20260812 00:11:12.509445 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:12.509646 7511 external_mini_cluster.cc:1428] Setting key 5d2f32d5a7f58ee327a7d0dc518b6909
I20260812 00:11:12.510125 15672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d383f419b6465e97d26615e03be9fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260812 00:11:12.510262 15672 leader_election.cc:302] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d383f419b6465e97d26615e03be9fa; no voters:
I20260812 00:11:12.510484 15672 leader_election.cc:288] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:12.510532 15679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:12.510716 15679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [term 1 LEADER]: Becoming Leader. State: Replica: 80d383f419b6465e97d26615e03be9fa, State: Running, Role: LEADER
I20260812 00:11:12.510775 15672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:12.510866 15679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d383f419b6465e97d26615e03be9fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } }
I20260812 00:11:12.511622 15680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80d383f419b6465e97d26615e03be9fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d383f419b6465e97d26615e03be9fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } } }
I20260812 00:11:12.511716 15680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:12.511889 15681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d383f419b6465e97d26615e03be9fa. Latest consensus state: current_term: 1 leader_uuid: "80d383f419b6465e97d26615e03be9fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d383f419b6465e97d26615e03be9fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33461 } } }
I20260812 00:11:12.511948 15691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:12.512008 15681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:12.512522 15691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:12.514299 15691 catalog_manager.cc:1347] Generated new cluster ID: 31bf737506ba4085beff34e74b885f16
I20260812 00:11:12.514357 15691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:11:12 dist-test-slave-z8x9 krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:11:12.525406 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:12.510174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:46959 (local address 127.0.0.1:33461)
0812 00:11:12.510366 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:12.510376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:12.510409 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:11:12.510562 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:12.510570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:12.510635 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:12.511647 (+ 1012us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:12.511772 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:12.513238 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:12.517245 (+ 4007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:12.517889 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:12.518009 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:12.521052 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:12.521058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:12.521058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:12.521072 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:12.523201 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:12.523730 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:12.523734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:12.523735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:12.523871 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:12.524188 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:12.524189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:12.524190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:12.524579 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:12.524757 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:12.524901 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 00:11:12.525029 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
I20260812 00:11:12.526734 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:12.536883 15691 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:12.537557 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:12.528331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:56063 (local address 127.0.0.1:33461)
0812 00:11:12.528371 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:12.528373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:12.529348 (+ 975us) server_negotiation.cc:407] Connection header received
0812 00:11:12.529386 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:12.529387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:12.529401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:12.529474 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:12.530130 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:12.531003 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:12.532206 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:12.532531 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:12.532595 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:12.534348 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:12.534349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:12.534350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:12.534358 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:12.536027 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:12.536552 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:12.536554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:12.536554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:12.536627 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:12.536910 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:12.536912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:12.536912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:12.537062 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:12.537217 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:12.537404 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 00:11:12.537472 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:11:12.537760 15691 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:12.543094 15691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80d383f419b6465e97d26615e03be9fa: Generated new TSK 0
I20260812 00:11:12.543977 15691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:12.550014 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15551
W20260812 00:11:12.564591 15527 connection.cc:535] client connection to 127.0.0.1:33461 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33461): Connection reset by peer (error 104)
2026-08-12T00:11:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1703 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:12 dist-test-slave-z8x9 krb5kdc[15721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:12 dist-test-slave-z8x9 krb5kdc[15721](info): set up 2 sockets
Aug 12 00:11:12 dist-test-slave-z8x9 krb5kdc[15721](info): commencing operation
krb5kdc: starting...
W20260812 00:11:13.940162 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.361s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:14 dist-test-slave-z8x9 krb5kdc[15721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:14.080700 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39321
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:14.169560 15750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:14.169855 15750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:14.169905 15750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:14.171152 15750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:14.171228 15750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:14.171257 15750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:14.171290 15750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:14.171314 15750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:14.172881 15750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39321
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:14.173228 15750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:14.173488 15750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:14.178910 15757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:14.179033 15750 server_base.cc:1034] running on GCE node
W20260812 00:11:14.178910 15758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:14.178969 15760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:14.179464 15750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:14.179808 15750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:14.180981 15750 hybrid_clock.cc:648] HybridClock initialized: now 1786493474180978 us; error 42 us; skew 500 ppm
Aug 12 00:11:14 dist-test-slave-z8x9 krb5kdc[15721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493474, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:14.189251 15750 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:14.189795 15750 webserver.cc:460] Webserver started at http://127.0.0.1:33285/ using document root <none> and password file <none>
I20260812 00:11:14.190074 15750 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:14.190146 15750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:14.190310 15750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:14.191406 15750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "80ed1a5199ac4798b84eec1a07a844dd"
format_stamp: "Formatted at 2026-08-12 00:11:14 on dist-test-slave-z8x9"
server_key: "f31c4af8f5cb011168c22a1c95c27bba"
server_key_iv: "be05c5f01526e4ccc3512b974fc2d2e2"
server_key_version: "encryptionkey@0"
I20260812 00:11:14.191778 15750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "80ed1a5199ac4798b84eec1a07a844dd"
format_stamp: "Formatted at 2026-08-12 00:11:14 on dist-test-slave-z8x9"
server_key: "f31c4af8f5cb011168c22a1c95c27bba"
server_key_iv: "be05c5f01526e4ccc3512b974fc2d2e2"
server_key_version: "encryptionkey@0"
I20260812 00:11:14.193158 15750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:11:14.194161 15775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:14.194381 15750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:14.194501 15750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "80ed1a5199ac4798b84eec1a07a844dd"
format_stamp: "Formatted at 2026-08-12 00:11:14 on dist-test-slave-z8x9"
server_key: "f31c4af8f5cb011168c22a1c95c27bba"
server_key_iv: "be05c5f01526e4ccc3512b974fc2d2e2"
server_key_version: "encryptionkey@0"
I20260812 00:11:14.194592 15750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:14.203771 15750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:14.206110 15750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:14.206332 15750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:14.211817 15750 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39321
I20260812 00:11:14.211864 15875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39321 every 8 connection(s)
I20260812 00:11:14.212276 15750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:14.213390 15876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:14.215979 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd: Bootstrap starting.
I20260812 00:11:14.216627 15876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:14.216943 15876 log.cc:824] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:14.217842 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd: No bootstrap required, opened a new log
I20260812 00:11:14.217840 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15750
I20260812 00:11:14.217986 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:14.218264 7511 external_mini_cluster.cc:1428] Setting key d93660d2dfe12b3b42e80036bfe85190
I20260812 00:11:14.219987 15876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ed1a5199ac4798b84eec1a07a844dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260812 00:11:14.220157 15876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:14.220225 15876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ed1a5199ac4798b84eec1a07a844dd, State: Initialized, Role: FOLLOWER
I20260812 00:11:14.220398 15876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ed1a5199ac4798b84eec1a07a844dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260812 00:11:14.220499 15876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:14.220544 15876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:14.220598 15876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:14.221222 15876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ed1a5199ac4798b84eec1a07a844dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260812 00:11:14.221355 15876 leader_election.cc:302] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80ed1a5199ac4798b84eec1a07a844dd; no voters:
I20260812 00:11:14.221557 15876 leader_election.cc:288] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:14.221585 15886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:14.221701 15886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [term 1 LEADER]: Becoming Leader. State: Replica: 80ed1a5199ac4798b84eec1a07a844dd, State: Running, Role: LEADER
I20260812 00:11:14.221843 15876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:14.221849 15886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ed1a5199ac4798b84eec1a07a844dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } }
I20260812 00:11:14.222294 15889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80ed1a5199ac4798b84eec1a07a844dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ed1a5199ac4798b84eec1a07a844dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } } }
I20260812 00:11:14.222548 15889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:14.222465 15888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80ed1a5199ac4798b84eec1a07a844dd. Latest consensus state: current_term: 1 leader_uuid: "80ed1a5199ac4798b84eec1a07a844dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80ed1a5199ac4798b84eec1a07a844dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39321 } } }
I20260812 00:11:14.222821 15888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:14.222995 15901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:14.223620 15901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:14.225654 15901 catalog_manager.cc:1347] Generated new cluster ID: 7a75e50683ef4087a0caacfe6bf7eb5d
I20260812 00:11:14.225858 15901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:14.226080 15885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:14.218881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:49469 (local address 127.0.0.1:39321)
0812 00:11:14.219073 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:14.219081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:14.219108 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:11:14.219990 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:14.219999 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:14.220058 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:14.220247 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:14.220525 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:14.222057 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:14.223140 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:14.223764 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:14.223874 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:14.224952 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:14.224957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:14.224957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:14.225237 (+ 280us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:14.225287 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:14.225302 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:14.225433 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:14.225518 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 00:11:14.225551 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260812 00:11:14.226977 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:14.232597 15885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:14.228592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:47297 (local address 127.0.0.1:39321)
0812 00:11:14.228628 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:14.228630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:14.228750 (+ 120us) server_negotiation.cc:407] Connection header received
0812 00:11:14.228858 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:14.228859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:14.228886 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:14.228916 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:14.229098 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:14.229899 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:14.231021 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:14.231380 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:14.231421 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:14.232303 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:14.232304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:14.232305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:14.232310 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:14.232325 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:14.232330 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:14.232363 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:14.232487 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 00:11:14.232504 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:11:14.260859 15901 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:14.261663 15901 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:14.271937 15901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80ed1a5199ac4798b84eec1a07a844dd: Generated new TSK 0
I20260812 00:11:14.272156 15901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:14.279422 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15750
W20260812 00:11:14.293752 15719 connection.cc:535] client connection to 127.0.0.1:39321 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39321): Connection reset by peer (error 104)
2026-08-12T00:11:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1729 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:14 dist-test-slave-z8x9 krb5kdc[15936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:14 dist-test-slave-z8x9 krb5kdc[15936](info): set up 2 sockets
Aug 12 00:11:14 dist-test-slave-z8x9 krb5kdc[15936](info): commencing operation
krb5kdc: starting...
W20260812 00:11:15.632315 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:15 dist-test-slave-z8x9 krb5kdc[15936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:11:15.769768 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:15.855068 15964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:15.855297 15964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:15.855322 15964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:15.856516 15964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:15.856566 15964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:15.856580 15964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:15.856593 15964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:15.856606 15964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:15.858183 15964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:15.858460 15964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:15.858704 15964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:15.863976 15973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:15.863969 15971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:15.863969 15976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:15.864239 15964 server_base.cc:1034] running on GCE node
I20260812 00:11:15.864635 15964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:15.864952 15964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:15.866171 15964 hybrid_clock.cc:648] HybridClock initialized: now 1786493475866134 us; error 54 us; skew 500 ppm
Aug 12 00:11:15 dist-test-slave-z8x9 krb5kdc[15936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493475, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:15.869626 15964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:11:15.870124 15964 webserver.cc:460] Webserver started at http://127.0.0.1:34721/ using document root <none> and password file <none>
I20260812 00:11:15.870373 15964 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:15.870446 15964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:15.870605 15964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:15.871692 15964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "a82acbb3692f4e199daea1f8b493abaf"
format_stamp: "Formatted at 2026-08-12 00:11:15 on dist-test-slave-z8x9"
server_key: "a14d31f042958900ec1621f72dc2f2b7"
server_key_iv: "f2dc8974af067508b39147076842619c"
server_key_version: "encryptionkey@0"
I20260812 00:11:15.872081 15964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "a82acbb3692f4e199daea1f8b493abaf"
format_stamp: "Formatted at 2026-08-12 00:11:15 on dist-test-slave-z8x9"
server_key: "a14d31f042958900ec1621f72dc2f2b7"
server_key_iv: "f2dc8974af067508b39147076842619c"
server_key_version: "encryptionkey@0"
I20260812 00:11:15.873922 15964 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 00:11:15.874852 15986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:15.875056 15964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:15.875181 15964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "a82acbb3692f4e199daea1f8b493abaf"
format_stamp: "Formatted at 2026-08-12 00:11:15 on dist-test-slave-z8x9"
server_key: "a14d31f042958900ec1621f72dc2f2b7"
server_key_iv: "f2dc8974af067508b39147076842619c"
server_key_version: "encryptionkey@0"
I20260812 00:11:15.875247 15964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:15.878774 15964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:15.881161 15964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:15.881347 15964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:15.885690 15964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33789
I20260812 00:11:15.885769 16092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33789 every 8 connection(s)
I20260812 00:11:15.886083 15964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:15.887127 16097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:15.889513 16097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf: Bootstrap starting.
I20260812 00:11:15.890105 16097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:15.890416 16097 log.cc:824] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:15.891278 16097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf: No bootstrap required, opened a new log
I20260812 00:11:15.893092 16097 raft_consensus.cc:348] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a82acbb3692f4e199daea1f8b493abaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33789 } }
I20260812 00:11:15.893376 16097 raft_consensus.cc:374] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:15.893464 16097 raft_consensus.cc:729] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a82acbb3692f4e199daea1f8b493abaf, State: Initialized, Role: FOLLOWER
I20260812 00:11:15.893651 16097 consensus_queue.cc:260] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a82acbb3692f4e199daea1f8b493abaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33789 } }
I20260812 00:11:15.893766 16097 raft_consensus.cc:388] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:15.893817 16097 raft_consensus.cc:482] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:15.893874 16097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:15.894472 16097 raft_consensus.cc:504] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a82acbb3692f4e199daea1f8b493abaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33789 } }
I20260812 00:11:15.894606 16097 leader_election.cc:302] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a82acbb3692f4e199daea1f8b493abaf; no voters:
I20260812 00:11:15.894817 16097 leader_election.cc:288] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:15.894845 16102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:15.894990 16102 raft_consensus.cc:686] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [term 1 LEADER]: Becoming Leader. State: Replica: a82acbb3692f4e199daea1f8b493abaf, State: Running, Role: LEADER
I20260812 00:11:15.895084 16102 consensus_queue.cc:237] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a82acbb3692f4e199daea1f8b493abaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33789 } }
I20260812 00:11:15.895118 16097 sys_catalog.cc:564] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:15.895541 16106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a82acbb3692f4e199daea1f8b493abaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a82acbb3692f4e199daea1f8b493abaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33789 } } }
I20260812 00:11:15.895675 16106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:15.895687 16104 sys_catalog.cc:455] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a82acbb3692f4e199daea1f8b493abaf. Latest consensus state: current_term: 1 leader_uuid: "a82acbb3692f4e199daea1f8b493abaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a82acbb3692f4e199daea1f8b493abaf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33789 } } }
I20260812 00:11:15.895915 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 15964
I20260812 00:11:15.895967 16104 sys_catalog.cc:458] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:15.896031 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:15.896106 16118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:15.896229 7511 external_mini_cluster.cc:1428] Setting key 8b671bda68bfa32ac63c0bdd07e8d89d
I20260812 00:11:15.896734 16118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:15.899226 16118 catalog_manager.cc:1347] Generated new cluster ID: 0fdd1aaa01634fc893b9ab97da2caff5
I20260812 00:11:15.899224 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:15.896924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:36317 (local address 127.0.0.1:33789)
0812 00:11:15.897109 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:15.897120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:15.897281 (+ 161us) server_negotiation.cc:407] Connection header received
0812 00:11:15.897470 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:15.897478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:15.897514 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:15.897712 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:15.898614 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:15.898618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:15.898618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:15.898623 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:15.898679 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:15.898691 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:15.898775 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:15.898996 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 00:11:15.899042 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260812 00:11:15.899317 16118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:15.900025 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:15.902797 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:15.901510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.69:54187 (local address 127.0.0.1:33789)
0812 00:11:15.901539 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:15.901541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:15.901805 (+ 264us) server_negotiation.cc:407] Connection header received
0812 00:11:15.901866 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:15.901867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:15.901873 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:15.901929 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:15.902579 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:15.902580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:15.902580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:15.902584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:15.902600 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:15.902604 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:15.902656 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:15.902726 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 00:11:15.902738 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 00:11:15.907521 16118 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:15.907724 16118 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:15.924518 16118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a82acbb3692f4e199daea1f8b493abaf: Generated new TSK 0
I20260812 00:11:15.924801 16118 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:15.928678 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 15964
2026-08-12T00:11:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:15 dist-test-slave-z8x9 krb5kdc[16160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:15 dist-test-slave-z8x9 krb5kdc[16160](info): set up 2 sockets
Aug 12 00:11:15 dist-test-slave-z8x9 krb5kdc[16160](info): commencing operation
krb5kdc: starting...
W20260812 00:11:17.265488 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:17 dist-test-slave-z8x9 krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:17.409324 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:32935
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:17.499668 16185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:17.499918 16185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:17.499943 16185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:17.501152 16185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:17.501250 16185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:17.501269 16185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:17.501287 16185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:17.501343 16185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:17.502928 16185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:32935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:17.503212 16185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:17.503466 16185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:17.508831 16199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:17.508882 16197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:17.508987 16185 server_base.cc:1034] running on GCE node
W20260812 00:11:17.508845 16196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:17.509393 16185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:17.509717 16185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:17.510907 16185 hybrid_clock.cc:648] HybridClock initialized: now 1786493477510883 us; error 45 us; skew 500 ppm
Aug 12 00:11:17 dist-test-slave-z8x9 krb5kdc[16160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493477, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:17.513715 16185 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:17.514194 16185 webserver.cc:460] Webserver started at http://127.7.85.254:46683/ using document root <none> and password file <none>
I20260812 00:11:17.514441 16185 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:17.514514 16185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:17.514678 16185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:17.515863 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a"
format_stamp: "Formatted at 2026-08-12 00:11:17 on dist-test-slave-z8x9"
server_key: "4bf4d456a41cce125ec1c9943173d246"
server_key_iv: "e25b01c2ac66417211bc4bcc8323f7a8"
server_key_version: "encryptionkey@0"
I20260812 00:11:17.516247 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a"
format_stamp: "Formatted at 2026-08-12 00:11:17 on dist-test-slave-z8x9"
server_key: "4bf4d456a41cce125ec1c9943173d246"
server_key_iv: "e25b01c2ac66417211bc4bcc8323f7a8"
server_key_version: "encryptionkey@0"
I20260812 00:11:17.517812 16185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:11:17.518817 16207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:17.519076 16185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:17.519200 16185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a"
format_stamp: "Formatted at 2026-08-12 00:11:17 on dist-test-slave-z8x9"
server_key: "4bf4d456a41cce125ec1c9943173d246"
server_key_iv: "e25b01c2ac66417211bc4bcc8323f7a8"
server_key_version: "encryptionkey@0"
I20260812 00:11:17.519285 16185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:17.541611 16185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:17.543877 16185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:17.544061 16185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:17.548699 16185 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:32935
I20260812 00:11:17.548796 16317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:32935 every 8 connection(s)
I20260812 00:11:17.549274 16185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:17.550266 16318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:17.552619 16318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a: Bootstrap starting.
I20260812 00:11:17.553516 16318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:17.553844 16318 log.cc:824] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:17.554632 16318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a: No bootstrap required, opened a new log
I20260812 00:11:17.556214 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16185
I20260812 00:11:17.556371 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:17.556365 16318 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 32935 } }
I20260812 00:11:17.556567 16318 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:17.556576 7511 external_mini_cluster.cc:1428] Setting key 61defe7c8e36e43874ebe3be1b59f86c
I20260812 00:11:17.556594 16318 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddbeee1d5df94c9ebfb7a4e445ed022a, State: Initialized, Role: FOLLOWER
I20260812 00:11:17.556789 16318 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 32935 } }
I20260812 00:11:17.556885 16318 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:17.556910 16318 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:17.557008 16318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:17.557946 16318 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 32935 } }
I20260812 00:11:17.558085 16318 leader_election.cc:302] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddbeee1d5df94c9ebfb7a4e445ed022a; no voters:
I20260812 00:11:17.558266 16318 leader_election.cc:288] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:17.558317 16334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:17.558455 16334 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [term 1 LEADER]: Becoming Leader. State: Replica: ddbeee1d5df94c9ebfb7a4e445ed022a, State: Running, Role: LEADER
I20260812 00:11:17.558625 16318 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:17.558607 16334 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 32935 } }
I20260812 00:11:17.559128 16336 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddbeee1d5df94c9ebfb7a4e445ed022a. Latest consensus state: current_term: 1 leader_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 32935 } } }
I20260812 00:11:17.559239 16336 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:17.559710 16335 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddbeee1d5df94c9ebfb7a4e445ed022a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 32935 } } }
I20260812 00:11:17.559866 16335 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:17.560073 16346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:17.560732 16346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:17.562644 16346 catalog_manager.cc:1347] Generated new cluster ID: 5f73501f338e48ea9acc6bf0e6009a23
I20260812 00:11:17.562739 16346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:11:17 dist-test-slave-z8x9 krb5kdc[16160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:11:17.570012 16333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:17.557115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50993 (local address 127.7.85.254:32935)
0812 00:11:17.557429 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:17.557441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:17.557498 (+ 57us) server_negotiation.cc:407] Connection header received
0812 00:11:17.557864 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:17.557870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:17.557918 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:17.558067 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:17.558909 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:17.560460 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:17.561728 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:17.562352 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:17.562473 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:17.565511 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:17.565517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:17.565517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:17.565533 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:17.567994 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:17.568542 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:17.568547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:17.568547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:17.568694 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:17.568958 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:17.568959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:17.568959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:17.569373 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:17.569567 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:17.569669 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 00:11:17.569788 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":149,"threads_started":1}
I20260812 00:11:17.571398 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:17.576124 16346 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:17.576898 16346 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:17.581364 16346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddbeee1d5df94c9ebfb7a4e445ed022a: Generated new TSK 0
I20260812 00:11:17.581607 16346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:17.582017 16333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:17.573049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45137 (local address 127.7.85.254:32935)
0812 00:11:17.573096 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:17.573098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:17.573152 (+ 54us) server_negotiation.cc:407] Connection header received
0812 00:11:17.573262 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:17.573263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:17.573275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:17.573333 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:17.574274 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:17.575152 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:17.576100 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:17.576460 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:17.576535 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:17.578140 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:17.578142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:17.578142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:17.578150 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:17.579870 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:17.580335 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:17.580337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:17.580337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:17.580589 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:17.581081 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:17.581082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:17.581082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:17.581280 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:17.581394 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:17.581853 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 00:11:17.581921 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":23}
I20260812 00:11:17.584187 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16185
W20260812 00:11:17.606412 16156 connection.cc:535] client connection to 127.7.85.254:32935 recv error: Network error: failed to read from TLS socket (remote: 127.7.85.254:32935): Connection reset by peer (error 104)
2026-08-12T00:11:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:17 dist-test-slave-z8x9 krb5kdc[16382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:17 dist-test-slave-z8x9 krb5kdc[16382](info): set up 2 sockets
Aug 12 00:11:17 dist-test-slave-z8x9 krb5kdc[16382](info): commencing operation
krb5kdc: starting...
W20260812 00:11:18.975205 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:19 dist-test-slave-z8x9 krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:19.120728 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:35185
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:35185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:19.212975 16414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:19.213268 16414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:19.213295 16414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:19.214552 16414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:19.214612 16414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:19.214628 16414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:19.214641 16414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:19.214653 16414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:19.216333 16414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:35185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:35185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:19.216710 16414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:19.216967 16414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:19.223217 16421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:19.223193 16424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:19.223310 16414 server_base.cc:1034] running on GCE node
W20260812 00:11:19.223193 16422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:19.224177 16414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:19.224687 16414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:19.225914 16414 hybrid_clock.cc:648] HybridClock initialized: now 1786493479225871 us; error 64 us; skew 500 ppm
Aug 12 00:11:19 dist-test-slave-z8x9 krb5kdc[16382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:19.234186 16414 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:19.234720 16414 webserver.cc:460] Webserver started at http://127.7.85.254:44275/ using document root <none> and password file <none>
I20260812 00:11:19.234965 16414 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:19.235016 16414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:19.235162 16414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:19.236333 16414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "a611f367dbd04519a3964923ec85c23a"
format_stamp: "Formatted at 2026-08-12 00:11:19 on dist-test-slave-z8x9"
server_key: "fb646f021512634eade69f02d8ffeb7c"
server_key_iv: "7c4a6072831083a19e921f30496b6271"
server_key_version: "encryptionkey@0"
I20260812 00:11:19.236722 16414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "a611f367dbd04519a3964923ec85c23a"
format_stamp: "Formatted at 2026-08-12 00:11:19 on dist-test-slave-z8x9"
server_key: "fb646f021512634eade69f02d8ffeb7c"
server_key_iv: "7c4a6072831083a19e921f30496b6271"
server_key_version: "encryptionkey@0"
I20260812 00:11:19.238348 16414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:11:19.239395 16439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:19.239935 16414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:19.240208 16414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "a611f367dbd04519a3964923ec85c23a"
format_stamp: "Formatted at 2026-08-12 00:11:19 on dist-test-slave-z8x9"
server_key: "fb646f021512634eade69f02d8ffeb7c"
server_key_iv: "7c4a6072831083a19e921f30496b6271"
server_key_version: "encryptionkey@0"
I20260812 00:11:19.240309 16414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:19.253834 16414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:19.256487 16414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:19.256698 16414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:19.263903 16414 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:35185
I20260812 00:11:19.263976 16526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:35185 every 8 connection(s)
I20260812 00:11:19.264372 16414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:19.265414 16530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:19.267665 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16414
I20260812 00:11:19.267797 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:19.267998 7511 external_mini_cluster.cc:1428] Setting key d14e45283f38496487ccb528f2d5c156
I20260812 00:11:19.268728 16530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a: Bootstrap starting.
I20260812 00:11:19.269552 16530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:19.269899 16530 log.cc:824] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:19.270845 16530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a: No bootstrap required, opened a new log
I20260812 00:11:19.273910 16530 raft_consensus.cc:348] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a611f367dbd04519a3964923ec85c23a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35185 } }
I20260812 00:11:19.274164 16530 raft_consensus.cc:374] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:19.274207 16530 raft_consensus.cc:729] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a611f367dbd04519a3964923ec85c23a, State: Initialized, Role: FOLLOWER
I20260812 00:11:19.274446 16530 consensus_queue.cc:260] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a611f367dbd04519a3964923ec85c23a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35185 } }
I20260812 00:11:19.274578 16530 raft_consensus.cc:388] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:19.274628 16530 raft_consensus.cc:482] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:19.274679 16530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:19.275514 16530 raft_consensus.cc:504] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a611f367dbd04519a3964923ec85c23a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35185 } }
I20260812 00:11:19.275629 16530 leader_election.cc:302] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a611f367dbd04519a3964923ec85c23a; no voters:
I20260812 00:11:19.275950 16530 leader_election.cc:288] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:19.276011 16537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:19.276331 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:19.268760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40101 (local address 127.7.85.254:35185)
0812 00:11:19.268962 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:19.268971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:19.269004 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:11:19.269719 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:19.269731 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:19.269800 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:19.269979 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:19.270351 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:19.271924 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:19.272988 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:19.273909 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:19.274120 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:19.275153 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:19.275157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:19.275157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:19.275374 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:19.275445 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:19.275476 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:19.275576 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:19.275786 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 00:11:19.275845 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260812 00:11:19.276368 16530 sys_catalog.cc:564] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:19.276366 16537 raft_consensus.cc:686] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [term 1 LEADER]: Becoming Leader. State: Replica: a611f367dbd04519a3964923ec85c23a, State: Running, Role: LEADER
I20260812 00:11:19.276580 16537 consensus_queue.cc:237] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a611f367dbd04519a3964923ec85c23a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35185 } }
I20260812 00:11:19.276892 16542 sys_catalog.cc:455] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a611f367dbd04519a3964923ec85c23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a611f367dbd04519a3964923ec85c23a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35185 } } }
I20260812 00:11:19.276969 16542 sys_catalog.cc:458] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:19.277632 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:19.278023 16541 sys_catalog.cc:455] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a611f367dbd04519a3964923ec85c23a. Latest consensus state: current_term: 1 leader_uuid: "a611f367dbd04519a3964923ec85c23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a611f367dbd04519a3964923ec85c23a" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35185 } } }
I20260812 00:11:19.278155 16541 sys_catalog.cc:458] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a [sys.catalog]: This master's current role is: LEADER
W20260812 00:11:19.278458 16570 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:19.278533 16570 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:19.278636 16553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:19.278870 16553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:19.282526 16553 catalog_manager.cc:1347] Generated new cluster ID: 597a44299ba04beaaf5287a72814ba65
I20260812 00:11:19.282591 16553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:19.283514 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:19.279319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56001 (local address 127.7.85.254:35185)
0812 00:11:19.279356 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:19.279358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:19.279513 (+ 155us) server_negotiation.cc:407] Connection header received
0812 00:11:19.279549 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:19.279550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:19.279561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:19.279622 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:19.279750 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:19.280455 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:19.281993 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:19.282334 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:19.282409 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:19.283241 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:19.283242 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:19.283243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:19.283248 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:19.283265 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:19.283269 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:19.283328 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:19.283430 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 00:11:19.283441 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:11:19.291483 16553 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:19.292317 16553 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:19.297080 16553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a611f367dbd04519a3964923ec85c23a: Generated new TSK 0
I20260812 00:11:19.297339 16553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:19.302246 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16414
W20260812 00:11:19.324240 16380 connection.cc:535] client connection to 127.7.85.254:35185 recv error: Network error: failed to read from TLS socket (remote: 127.7.85.254:35185): Connection reset by peer (error 104)
2026-08-12T00:11:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1718 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:19 dist-test-slave-z8x9 krb5kdc[16590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:19 dist-test-slave-z8x9 krb5kdc[16590](info): set up 2 sockets
Aug 12 00:11:19 dist-test-slave-z8x9 krb5kdc[16590](info): commencing operation
krb5kdc: starting...
W20260812 00:11:20.653992 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:20 dist-test-slave-z8x9 krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:20.802027 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41463
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:37461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:41463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:20.893301 16608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:20.893564 16608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:20.893604 16608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:20.894788 16608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:20.894868 16608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:20.894893 16608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:20.894922 16608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:20.894953 16608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:20.896487 16608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:37461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:41463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:41463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:20.896790 16608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:20.897032 16608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:20.902774 16623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:20.902770 16628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:20.902770 16625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:20.903211 16608 server_base.cc:1034] running on GCE node
I20260812 00:11:20.903512 16608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:20.903810 16608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:20.904955 16608 hybrid_clock.cc:648] HybridClock initialized: now 1786493480904937 us; error 28 us; skew 500 ppm
Aug 12 00:11:20 dist-test-slave-z8x9 krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:20.908037 16608 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:20.908612 16608 webserver.cc:460] Webserver started at http://127.7.85.254:34543/ using document root <none> and password file <none>
I20260812 00:11:20.908869 16608 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:20.908941 16608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:20.909097 16608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:20.910250 16608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "54ff1567db1a47bb9b1e603ffb000061"
format_stamp: "Formatted at 2026-08-12 00:11:20 on dist-test-slave-z8x9"
server_key: "0bb192acc3d0f55c05d265f134a73cc9"
server_key_iv: "3d1976287c33d47be8d1ed3a79b4be4c"
server_key_version: "encryptionkey@0"
I20260812 00:11:20.910637 16608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "54ff1567db1a47bb9b1e603ffb000061"
format_stamp: "Formatted at 2026-08-12 00:11:20 on dist-test-slave-z8x9"
server_key: "0bb192acc3d0f55c05d265f134a73cc9"
server_key_iv: "3d1976287c33d47be8d1ed3a79b4be4c"
server_key_version: "encryptionkey@0"
I20260812 00:11:20.912168 16608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 00:11:20.913125 16635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:20.913391 16608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:20.913517 16608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "54ff1567db1a47bb9b1e603ffb000061"
format_stamp: "Formatted at 2026-08-12 00:11:20 on dist-test-slave-z8x9"
server_key: "0bb192acc3d0f55c05d265f134a73cc9"
server_key_iv: "3d1976287c33d47be8d1ed3a79b4be4c"
server_key_version: "encryptionkey@0"
I20260812 00:11:20.913609 16608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:20.917313 16608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:20.919531 16608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:20.919708 16608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:20.924152 16608 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:41463
I20260812 00:11:20.924191 16721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:41463 every 8 connection(s)
I20260812 00:11:20.924716 16608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:20.925912 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:20.928315 16722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061: Bootstrap starting.
I20260812 00:11:20.929008 16722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:20.929140 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16608
I20260812 00:11:20.929401 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:20.929407 16722 log.cc:824] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:20.929726 7511 external_mini_cluster.cc:1428] Setting key 219bb886e9fadf762ff84fdb1e8d16e3
I20260812 00:11:20.930603 16722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061: No bootstrap required, opened a new log
I20260812 00:11:20.933020 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:20.930347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37731 (local address 127.7.85.254:41463)
0812 00:11:20.930545 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:20.930556 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:20.930596 (+ 40us) server_negotiation.cc:407] Connection header received
0812 00:11:20.931349 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:20.931358 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:20.931389 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:20.931565 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:20.932398 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:20.932410 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:20.932410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:20.932414 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:20.932458 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:20.932469 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:20.932569 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:20.932609 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 00:11:20.932654 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260812 00:11:20.932986 16722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff1567db1a47bb9b1e603ffb000061" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41463 } }
I20260812 00:11:20.933161 16722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:20.933239 16722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff1567db1a47bb9b1e603ffb000061, State: Initialized, Role: FOLLOWER
I20260812 00:11:20.933367 16722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff1567db1a47bb9b1e603ffb000061" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41463 } }
I20260812 00:11:20.933475 16722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:20.933535 16722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:20.933588 16722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:20.934196 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:20.934425 16722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff1567db1a47bb9b1e603ffb000061" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41463 } }
I20260812 00:11:20.934551 16722 leader_election.cc:302] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54ff1567db1a47bb9b1e603ffb000061; no voters:
I20260812 00:11:20.934763 16722 leader_election.cc:288] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:20.934827 16732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:20.935009 16732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [term 1 LEADER]: Becoming Leader. State: Replica: 54ff1567db1a47bb9b1e603ffb000061, State: Running, Role: LEADER
I20260812 00:11:20.935043 16722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:20.935194 16732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff1567db1a47bb9b1e603ffb000061" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41463 } }
I20260812 00:11:20.935570 16732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54ff1567db1a47bb9b1e603ffb000061. Latest consensus state: current_term: 1 leader_uuid: "54ff1567db1a47bb9b1e603ffb000061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff1567db1a47bb9b1e603ffb000061" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41463 } } }
I20260812 00:11:20.935724 16732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:20.935585 16733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54ff1567db1a47bb9b1e603ffb000061" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff1567db1a47bb9b1e603ffb000061" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 41463 } } }
I20260812 00:11:20.936084 16733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:20.936094 16750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:20.938107 16750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:20.938333 16728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:20.936103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42957 (local address 127.7.85.254:41463)
0812 00:11:20.936137 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:20.936139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:20.936148 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:20.936169 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:20.936170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:20.936175 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:20.936217 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:20.937908 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:20.937910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:20.937911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:20.937916 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:20.937996 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:20.938009 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:20.938096 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:20.938246 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 00:11:20.938261 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 00:11:20.940032 16750 catalog_manager.cc:1347] Generated new cluster ID: a996d6779e5a4da6ba78f49641420086
I20260812 00:11:20.940095 16750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:20.947081 16750 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:20.947285 16750 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:20.954579 16750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54ff1567db1a47bb9b1e603ffb000061: Generated new TSK 0
I20260812 00:11:20.954810 16750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:20.959969 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16608
2026-08-12T00:11:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1652 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:21 dist-test-slave-z8x9 krb5kdc[16778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:21 dist-test-slave-z8x9 krb5kdc[16778](info): set up 2 sockets
Aug 12 00:11:21 dist-test-slave-z8x9 krb5kdc[16778](info): commencing operation
krb5kdc: starting...
W20260812 00:11:22.298374 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:22 dist-test-slave-z8x9 krb5kdc[16778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:22.442095 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36813
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:38057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:22.530355 16800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:22.530632 16800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:22.530659 16800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:22.531936 16800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:22.531998 16800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:22.532014 16800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:22.532027 16800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:22.532039 16800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:22.533666 16800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:38057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:22.533959 16800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:22.534241 16800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:22.539690 16815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:22.539690 16811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:22.539746 16810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:22.540122 16800 server_base.cc:1034] running on GCE node
I20260812 00:11:22.540427 16800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:22.540767 16800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:22.541942 16800 hybrid_clock.cc:648] HybridClock initialized: now 1786493482541918 us; error 36 us; skew 500 ppm
Aug 12 00:11:22 dist-test-slave-z8x9 krb5kdc[16778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:22.545058 16800 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:22.545913 16800 webserver.cc:460] Webserver started at http://127.7.85.254:43297/ using document root <none> and password file <none>
I20260812 00:11:22.546126 16800 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:22.546171 16800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:22.546298 16800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:22.547391 16800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "97f6d880230d42cba18d7f2e3f24c510"
format_stamp: "Formatted at 2026-08-12 00:11:22 on dist-test-slave-z8x9"
server_key: "993ec8db595f05d71d1427725c46b05c"
server_key_iv: "cd0a24220361d1041ec26c23e0bf3d8e"
server_key_version: "encryptionkey@0"
I20260812 00:11:22.547708 16800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "97f6d880230d42cba18d7f2e3f24c510"
format_stamp: "Formatted at 2026-08-12 00:11:22 on dist-test-slave-z8x9"
server_key: "993ec8db595f05d71d1427725c46b05c"
server_key_iv: "cd0a24220361d1041ec26c23e0bf3d8e"
server_key_version: "encryptionkey@0"
I20260812 00:11:22.549243 16800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 00:11:22.550097 16828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:22.550426 16800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:22.550570 16800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "97f6d880230d42cba18d7f2e3f24c510"
format_stamp: "Formatted at 2026-08-12 00:11:22 on dist-test-slave-z8x9"
server_key: "993ec8db595f05d71d1427725c46b05c"
server_key_iv: "cd0a24220361d1041ec26c23e0bf3d8e"
server_key_version: "encryptionkey@0"
I20260812 00:11:22.550660 16800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:22.574942 16800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:22.577235 16800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:22.577411 16800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:22.582181 16800 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:36813
I20260812 00:11:22.582226 16924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:36813 every 8 connection(s)
I20260812 00:11:22.582581 16800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:22.583540 16925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:22.586202 16925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510: Bootstrap starting.
I20260812 00:11:22.586905 16925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:22.587167 16925 log.cc:824] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:22.587913 16925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510: No bootstrap required, opened a new log
I20260812 00:11:22.589041 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 16800
I20260812 00:11:22.589188 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:22.589452 7511 external_mini_cluster.cc:1428] Setting key b314e2f173752ffd373e0d58766c9a76
I20260812 00:11:22.590272 16925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f6d880230d42cba18d7f2e3f24c510" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36813 } }
I20260812 00:11:22.590425 16925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:22.590446 16925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f6d880230d42cba18d7f2e3f24c510, State: Initialized, Role: FOLLOWER
I20260812 00:11:22.590590 16925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f6d880230d42cba18d7f2e3f24c510" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36813 } }
I20260812 00:11:22.590698 16925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:22.590744 16925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:22.590854 16925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:22.591715 16925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f6d880230d42cba18d7f2e3f24c510" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36813 } }
I20260812 00:11:22.591817 16925 leader_election.cc:302] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97f6d880230d42cba18d7f2e3f24c510; no voters:
I20260812 00:11:22.592007 16925 leader_election.cc:288] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:22.592069 16933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:22.592255 16933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [term 1 LEADER]: Becoming Leader. State: Replica: 97f6d880230d42cba18d7f2e3f24c510, State: Running, Role: LEADER
I20260812 00:11:22.592334 16925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:22.592463 16933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f6d880230d42cba18d7f2e3f24c510" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36813 } }
I20260812 00:11:22.592880 16933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97f6d880230d42cba18d7f2e3f24c510. Latest consensus state: current_term: 1 leader_uuid: "97f6d880230d42cba18d7f2e3f24c510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f6d880230d42cba18d7f2e3f24c510" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36813 } } }
I20260812 00:11:22.592893 16935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97f6d880230d42cba18d7f2e3f24c510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97f6d880230d42cba18d7f2e3f24c510" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36813 } } }
I20260812 00:11:22.593050 16935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:22.593272 16933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:22.593422 16947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:22.593993 16947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:22.595821 16947 catalog_manager.cc:1347] Generated new cluster ID: dc2a03b3d1cc4767b351ba27e204e352
I20260812 00:11:22.595876 16947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 00:11:22 dist-test-slave-z8x9 krb5kdc[16778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:11:22.602165 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:22.590079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47725 (local address 127.7.85.254:36813)
0812 00:11:22.590267 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:22.590277 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:22.590312 (+ 35us) server_negotiation.cc:407] Connection header received
0812 00:11:22.590449 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:22.590456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:22.590522 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:22.590673 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:22.591538 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:22.592927 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:22.594113 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:22.594709 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:22.594819 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:22.598067 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:22.598071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:22.598072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:22.598086 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:22.600273 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:22.600685 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:22.600689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:22.600689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:22.600827 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:22.601038 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:22.601039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:22.601040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:22.601424 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:22.601594 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:22.601705 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 00:11:22.601873 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260812 00:11:22.603215 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:22.612547 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:22.604723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35195 (local address 127.7.85.254:36813)
0812 00:11:22.604760 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:22.604762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:22.605090 (+ 328us) server_negotiation.cc:407] Connection header received
0812 00:11:22.605136 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:22.605138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:22.605170 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:22.605266 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:22.605908 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:22.606714 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:22.607708 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:22.608037 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:22.608114 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:22.609591 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:22.609593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:22.609593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:22.609601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:22.611306 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:22.611720 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:22.611721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:22.611722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:22.611791 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:22.611990 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:22.611991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:22.611991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:22.612139 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:22.612264 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:22.612355 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 00:11:22.612441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:11:22.620321 16947 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:22.621073 16947 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:22.624454 16947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97f6d880230d42cba18d7f2e3f24c510: Generated new TSK 0
I20260812 00:11:22.624843 16947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:22.632427 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 16800
2026-08-12T00:11:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:22 dist-test-slave-z8x9 krb5kdc[16985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:22 dist-test-slave-z8x9 krb5kdc[16985](info): set up 2 sockets
Aug 12 00:11:22 dist-test-slave-z8x9 krb5kdc[16985](info): commencing operation
krb5kdc: starting...
W20260812 00:11:24.006251 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:24 dist-test-slave-z8x9 krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:24.161573 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:44465
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:36277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:44465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:24.253212 17010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:24.253504 17010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:24.253548 17010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:24.254748 17010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:24.254825 17010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:24.254850 17010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:24.254879 17010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:24.254910 17010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:24.256471 17010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:36277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:44465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:24.256775 17010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:24.257025 17010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:24.262070 17024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:24.262046 17031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:24.262046 17022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:24.262291 17010 server_base.cc:1034] running on GCE node
I20260812 00:11:24.262593 17010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:24.262897 17010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:24.264101 17010 hybrid_clock.cc:648] HybridClock initialized: now 1786493484264054 us; error 59 us; skew 500 ppm
Aug 12 00:11:24 dist-test-slave-z8x9 krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:24.273092 17010 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:24.274113 17010 webserver.cc:460] Webserver started at http://127.7.85.254:40441/ using document root <none> and password file <none>
I20260812 00:11:24.274394 17010 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:24.274468 17010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:24.274629 17010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:24.275763 17010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "01e53ebe7e094b308335f0292c5d27e0"
format_stamp: "Formatted at 2026-08-12 00:11:24 on dist-test-slave-z8x9"
server_key: "7040995c24cdcda8fd4b1daf732b3b2b"
server_key_iv: "11a914a7c31ce4ad3dbd47de1e3c3435"
server_key_version: "encryptionkey@0"
I20260812 00:11:24.276154 17010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "01e53ebe7e094b308335f0292c5d27e0"
format_stamp: "Formatted at 2026-08-12 00:11:24 on dist-test-slave-z8x9"
server_key: "7040995c24cdcda8fd4b1daf732b3b2b"
server_key_iv: "11a914a7c31ce4ad3dbd47de1e3c3435"
server_key_version: "encryptionkey@0"
I20260812 00:11:24.277670 17010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 00:11:24.278513 17044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:24.278692 17010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:24.278805 17010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "01e53ebe7e094b308335f0292c5d27e0"
format_stamp: "Formatted at 2026-08-12 00:11:24 on dist-test-slave-z8x9"
server_key: "7040995c24cdcda8fd4b1daf732b3b2b"
server_key_iv: "11a914a7c31ce4ad3dbd47de1e3c3435"
server_key_version: "encryptionkey@0"
I20260812 00:11:24.278885 17010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:24.285930 17010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:24.288134 17010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:24.288341 17010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:24.292660 17010 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:44465
I20260812 00:11:24.292693 17144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:44465 every 8 connection(s)
I20260812 00:11:24.293061 17010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:24.294010 17146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:24.296381 17146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0: Bootstrap starting.
I20260812 00:11:24.297088 17146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:24.297528 17146 log.cc:824] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:24.298118 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17010
I20260812 00:11:24.298235 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:24.298470 7511 external_mini_cluster.cc:1428] Setting key 5a6ab3760ee7e782d761378559011101
I20260812 00:11:24.298468 17146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0: No bootstrap required, opened a new log
I20260812 00:11:24.300400 17146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e53ebe7e094b308335f0292c5d27e0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 44465 } }
I20260812 00:11:24.300546 17146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:24.300570 17146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e53ebe7e094b308335f0292c5d27e0, State: Initialized, Role: FOLLOWER
I20260812 00:11:24.300726 17146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e53ebe7e094b308335f0292c5d27e0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 44465 } }
I20260812 00:11:24.300853 17146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:24.300901 17146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:24.301005 17146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:24.301858 17146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e53ebe7e094b308335f0292c5d27e0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 44465 } }
I20260812 00:11:24.301978 17146 leader_election.cc:302] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e53ebe7e094b308335f0292c5d27e0; no voters:
I20260812 00:11:24.302150 17146 leader_election.cc:288] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:24.302251 17163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:24.302407 17146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:24.302507 17163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [term 1 LEADER]: Becoming Leader. State: Replica: 01e53ebe7e094b308335f0292c5d27e0, State: Running, Role: LEADER
I20260812 00:11:24.302662 17163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e53ebe7e094b308335f0292c5d27e0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 44465 } }
I20260812 00:11:24.303515 17165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e53ebe7e094b308335f0292c5d27e0. Latest consensus state: current_term: 1 leader_uuid: "01e53ebe7e094b308335f0292c5d27e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e53ebe7e094b308335f0292c5d27e0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 44465 } } }
I20260812 00:11:24.303649 17165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:24.303694 17164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e53ebe7e094b308335f0292c5d27e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e53ebe7e094b308335f0292c5d27e0" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 44465 } } }
I20260812 00:11:24.303771 17164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0 [sys.catalog]: This master's current role is: LEADER
W20260812 00:11:24.304391 17186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:24.304451 17186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:24.304718 17184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:24.305006 17184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:24.307413 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:24.299031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34345 (local address 127.7.85.254:44465)
0812 00:11:24.299261 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:24.299270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:24.299298 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:11:24.300422 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:24.300431 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:24.300494 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:24.300696 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:24.301008 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:24.302787 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:24.304095 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:24.304739 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:24.305796 (+ 1057us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:24.306335 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:24.306341 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:24.306341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:24.306623 (+ 282us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:24.306679 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:24.306699 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:24.306802 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:24.306945 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 00:11:24.307050 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260812 00:11:24.308264 17184 catalog_manager.cc:1347] Generated new cluster ID: 7fa56544d0be46b89ba15b2041e6ce61
I20260812 00:11:24.308365 17184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:24.308713 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:24.313515 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:24.310313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59327 (local address 127.7.85.254:44465)
0812 00:11:24.310358 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:24.310361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:24.310464 (+ 103us) server_negotiation.cc:407] Connection header received
0812 00:11:24.310529 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:24.310530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:24.310542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:24.310600 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:24.310746 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:24.311259 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:24.312159 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:24.312376 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:24.312459 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:24.313231 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:24.313232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:24.313233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:24.313240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:24.313256 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:24.313261 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:24.313316 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:24.313377 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 00:11:24.313406 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:11:24.318966 17184 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:24.319722 17184 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:24.328293 17184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e53ebe7e094b308335f0292c5d27e0: Generated new TSK 0
I20260812 00:11:24.328542 17184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:24.336951 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17010
2026-08-12T00:11:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1712 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:24 dist-test-slave-z8x9 krb5kdc[17213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:24 dist-test-slave-z8x9 krb5kdc[17213](info): set up 2 sockets
Aug 12 00:11:24 dist-test-slave-z8x9 krb5kdc[17213](info): commencing operation
krb5kdc: starting...
W20260812 00:11:25.725694 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:25 dist-test-slave-z8x9 krb5kdc[17213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:25.865804 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:35909
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:35909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:25.954219 17236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:25.954454 17236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:25.954479 17236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:25.955703 17236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:25.955760 17236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:25.955775 17236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:25.955787 17236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:25.955801 17236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:25.957404 17236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:35909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:35909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:25.957700 17236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:25.957955 17236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:25.963078 17244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:25.963198 17250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:25.963335 17245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:25.963627 17236 server_base.cc:1034] running on GCE node
I20260812 00:11:25.963938 17236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:25.964293 17236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:25.965448 17236 hybrid_clock.cc:648] HybridClock initialized: now 1786493485965419 us; error 42 us; skew 500 ppm
Aug 12 00:11:25 dist-test-slave-z8x9 krb5kdc[17213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:25.968353 17236 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:25.968907 17236 webserver.cc:460] Webserver started at http://127.7.85.254:40103/ using document root <none> and password file <none>
I20260812 00:11:25.969230 17236 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:25.969292 17236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:25.969448 17236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:25.970563 17236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379"
format_stamp: "Formatted at 2026-08-12 00:11:25 on dist-test-slave-z8x9"
server_key: "ecb846698915e91f1ac689bb561e4383"
server_key_iv: "82b918371c97c8066e8bf0b70cf66671"
server_key_version: "encryptionkey@0"
I20260812 00:11:25.970956 17236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379"
format_stamp: "Formatted at 2026-08-12 00:11:25 on dist-test-slave-z8x9"
server_key: "ecb846698915e91f1ac689bb561e4383"
server_key_iv: "82b918371c97c8066e8bf0b70cf66671"
server_key_version: "encryptionkey@0"
I20260812 00:11:25.972602 17236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:11:25.973610 17263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:25.973904 17236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:25.973994 17236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379"
format_stamp: "Formatted at 2026-08-12 00:11:25 on dist-test-slave-z8x9"
server_key: "ecb846698915e91f1ac689bb561e4383"
server_key_iv: "82b918371c97c8066e8bf0b70cf66671"
server_key_version: "encryptionkey@0"
I20260812 00:11:25.974097 17236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:25.978014 17236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:25.980446 17236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:25.980628 17236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:25.985526 17236 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:35909
I20260812 00:11:25.985597 17364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:35909 every 8 connection(s)
I20260812 00:11:25.985944 17236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:25.987066 17366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:25.989499 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379: Bootstrap starting.
I20260812 00:11:25.990060 17366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:25.990321 17366 log.cc:824] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:25.991081 17366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379: No bootstrap required, opened a new log
I20260812 00:11:25.992057 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17236
I20260812 00:11:25.992200 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:25.992522 7511 external_mini_cluster.cc:1428] Setting key c6926c43a33fc33530eca3917c3469a9
I20260812 00:11:25.993271 17366 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35909 } }
I20260812 00:11:25.993481 17366 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:11:25.993527 17366 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bfaae17dce4ec1b5aa5ca2fe0bf379, State: Initialized, Role: FOLLOWER
I20260812 00:11:25.993660 17366 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35909 } }
I20260812 00:11:25.993741 17366 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:11:25.993779 17366 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:11:25.993839 17366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:25.995033 17366 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35909 } }
I20260812 00:11:25.995148 17366 leader_election.cc:302] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8bfaae17dce4ec1b5aa5ca2fe0bf379; no voters:
I20260812 00:11:25.995182 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:25.993232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49075 (local address 127.7.85.254:35909)
0812 00:11:25.993471 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:25.993481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:25.993518 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:11:25.993640 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:25.993647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:25.993691 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:25.993848 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:25.994651 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:25.994656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:25.994656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:25.994661 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:25.994718 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:25.994736 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:25.994816 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:25.994937 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 00:11:25.994984 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
I20260812 00:11:25.995332 17366 leader_election.cc:288] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:11:25.995380 17381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:25.995548 17381 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [term 1 LEADER]: Becoming Leader. State: Replica: d8bfaae17dce4ec1b5aa5ca2fe0bf379, State: Running, Role: LEADER
I20260812 00:11:25.995638 17366 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:25.995695 17381 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35909 } }
I20260812 00:11:25.996120 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:25.996392 17382 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35909 } } }
I20260812 00:11:25.996466 17382 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:25.996632 17384 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8bfaae17dce4ec1b5aa5ca2fe0bf379. Latest consensus state: current_term: 1 leader_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bfaae17dce4ec1b5aa5ca2fe0bf379" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 35909 } } }
I20260812 00:11:25.996706 17384 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:25.997263 17395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:25.997838 17395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:25.999368 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:25.997995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54277 (local address 127.7.85.254:35909)
0812 00:11:25.998052 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:25.998054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:25.998260 (+ 206us) server_negotiation.cc:407] Connection header received
0812 00:11:25.998362 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:25.998363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:25.998368 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:25.998437 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:25.999122 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:25.999123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:25.999124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:11:25.999128 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:25.999146 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:11:25.999150 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:25.999213 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:25.999284 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 00:11:25.999300 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 00:11:25.999717 17395 catalog_manager.cc:1347] Generated new cluster ID: d1e6602dbc994b168eebb8eec117c0f4
I20260812 00:11:25.999768 17395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:26.011426 17395 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:26.011644 17395 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:26.017041 17395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8bfaae17dce4ec1b5aa5ca2fe0bf379: Generated new TSK 0
I20260812 00:11:26.017340 17395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:11:26.021687 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17236
2026-08-12T00:11:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1671 ms)
[----------] 18 tests from AuthTokenIssuingTest (30598 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:26 dist-test-slave-z8x9 krb5kdc[17429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:26 dist-test-slave-z8x9 krb5kdc[17429](info): set up 2 sockets
Aug 12 00:11:26 dist-test-slave-z8x9 krb5kdc[17429](info): commencing operation
krb5kdc: starting...
W20260812 00:11:27.364794 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:27.509708 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40841
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:40841,127.7.85.253:41255,127.7.85.252:38951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:27.597515 17449 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:11:27.597791 17449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:27.597824 17449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:27.597869 17449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:27.599169 17449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:27.599241 17449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:27.599277 17449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:27.599327 17449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:27.599354 17449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:27.600994 17449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:40841,127.7.85.253:41255,127.7.85.252:38951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:40841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:27.601344 17449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:27.601620 17449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:27.607725 17464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:27.607764 17459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:27.607795 17458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:27.609270 17449 server_base.cc:1034] running on GCE node
I20260812 00:11:27.609555 17449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:27.609879 17449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:27.611078 17449 hybrid_clock.cc:648] HybridClock initialized: now 1786493487611050 us; error 38 us; skew 500 ppm
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:27.614197 17449 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:27.614713 17449 webserver.cc:460] Webserver started at http://127.7.85.254:36507/ using document root <none> and password file <none>
I20260812 00:11:27.614979 17449 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:27.615079 17449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:27.615242 17449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:27.616441 17449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "4ef67f42a8e345899492ce76c4a41ab5"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "497cbfe37453c533f27ea64b7a7397d2"
server_key_iv: "f0abb9f03e7b99b550a5452cee8d0a3d"
server_key_version: "encryptionkey@0"
I20260812 00:11:27.616823 17449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "4ef67f42a8e345899492ce76c4a41ab5"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "497cbfe37453c533f27ea64b7a7397d2"
server_key_iv: "f0abb9f03e7b99b550a5452cee8d0a3d"
server_key_version: "encryptionkey@0"
I20260812 00:11:27.618412 17449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 00:11:27.619410 17477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:27.619675 17449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:11:27.619800 17449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "4ef67f42a8e345899492ce76c4a41ab5"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "497cbfe37453c533f27ea64b7a7397d2"
server_key_iv: "f0abb9f03e7b99b550a5452cee8d0a3d"
server_key_version: "encryptionkey@0"
I20260812 00:11:27.619885 17449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:27.643962 17449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:27.646545 17449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:27.646725 17449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:27.651289 17449 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:40841
I20260812 00:11:27.651342 17583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:40841 every 8 connection(s)
I20260812 00:11:27.651736 17449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:27.652763 17584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:27.654749 17584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:27.656803 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17449
I20260812 00:11:27.656975 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:27.657276 7511 external_mini_cluster.cc:1428] Setting key 635695c95e79ef19d8548c615059bdf8
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.253@KRBTEST.COM; defaulting to no policy
I20260812 00:11:27.667501 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.655692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.7.85.254:40841)
0812 00:11:27.655934 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:27.655947 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:27.656006 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:11:27.656612 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:27.656615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:27.656641 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:27.656781 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:27.657610 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.659104 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.660584 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.660930 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.661033 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.663714 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:27.663723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:27.663723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:27.663731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:27.665621 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.666081 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.666084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.666085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.666177 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.666442 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.666443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.666443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.666749 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:27.666853 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:27.667066 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 00:11:27.667207 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":90,"threads_started":1}
I20260812 00:11:27.667501 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.655626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40841 (local address 127.0.0.1:47914)
0812 00:11:27.655891 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.655932 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:27.656426 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:27.656826 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:27.656830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:27.657273 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:27.657499 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.657504 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.659255 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.659257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.660451 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.660455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.661055 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.661057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.661143 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.661810 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:27.661820 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:27.663602 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:27.665742 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.665749 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.665752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.665970 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.666271 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.666272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.666272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.666367 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.666830 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:27.666837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:27.666917 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 00:11:27.667028 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 00:11:27.667294 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":99,"thread_start_us":110,"threads_started":1}
I20260812 00:11:27.668768 17584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:27.669601 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.669428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36298)
0812 00:11:27.669500 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.669562 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 00:11:27.669883 17483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
I20260812 00:11:27.670680 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.670568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36310)
0812 00:11:27.670620 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.670652 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 00:11:27.670814 17584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:41255: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab.
Entry for principal kudu/127.7.85.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab.
I20260812 00:11:27.693382 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:41255
--webserver_interface=127.7.85.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab
--principal=kudu/127.7.85.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:40841,127.7.85.253:41255,127.7.85.252:38951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:11:27.720129 17584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } attempt: 1
I20260812 00:11:27.720715 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.720569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36314)
0812 00:11:27.720640 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.720684 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 00:11:27.721155 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.721051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36330)
0812 00:11:27.721101 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.721127 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 00:11:27.721308 17584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:41255: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
W20260812 00:11:27.788373 17600 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:11:27.788681 17600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:27.788715 17600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:27.788762 17600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 00:11:27.789503 17584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } attempt: 2
I20260812 00:11:27.790030 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.789881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36344)
0812 00:11:27.789954 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.789997 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 00:11:27.790354 17600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:27.790413 17600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:27.790439 17600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:27.790485 17600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260812 00:11:27.790488 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.790363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36354)
0812 00:11:27.790425 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.790460 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 00:11:27.790506 17600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:11:27.790613 17584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:41255: Network error: Client connection negotiation failed: client connection to 127.7.85.253:41255: connect: Connection refused (error 111)
I20260812 00:11:27.792236 17600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:40841,127.7.85.253:41255,127.7.85.252:38951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab
--principal=kudu/127.7.85.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:41255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.85.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:27.792519 17600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:27.792796 17600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:27.798449 17613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:27.798646 17614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:27.798646 17616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:27.798874 17600 server_base.cc:1034] running on GCE node
I20260812 00:11:27.799569 17600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:27.800185 17600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:27.801373 17600 hybrid_clock.cc:648] HybridClock initialized: now 1786493487801317 us; error 120 us; skew 500 ppm
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:27.804719 17600 init.cc:377] Logged in from keytab as kudu/127.7.85.253@KRBTEST.COM (short username kudu)
I20260812 00:11:27.805361 17600 webserver.cc:460] Webserver started at http://127.7.85.253:41971/ using document root <none> and password file <none>
I20260812 00:11:27.805615 17600 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:27.805691 17600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:27.805858 17600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:27.807088 17600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data/instance:
uuid: "d77012262040496395638848b3e09da7"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "7af5602d7ff12e8ad6585e13ea5923b7"
server_key_iv: "41eb4e2b3354362da6c0b3a031d39eb9"
server_key_version: "encryptionkey@0"
I20260812 00:11:27.807518 17600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal/instance:
uuid: "d77012262040496395638848b3e09da7"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "7af5602d7ff12e8ad6585e13ea5923b7"
server_key_iv: "41eb4e2b3354362da6c0b3a031d39eb9"
server_key_version: "encryptionkey@0"
I20260812 00:11:27.809577 17600 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 00:11:27.810556 17630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:27.810864 17600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:27.810995 17600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal
uuid: "d77012262040496395638848b3e09da7"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "7af5602d7ff12e8ad6585e13ea5923b7"
server_key_iv: "41eb4e2b3354362da6c0b3a031d39eb9"
server_key_version: "encryptionkey@0"
I20260812 00:11:27.811088 17600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:27.838568 17600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:27.841331 17600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:27.841516 17600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:27.846683 17600 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.253:41255
I20260812 00:11:27.846774 17729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.253:41255 every 8 connection(s)
I20260812 00:11:27.847175 17600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
I20260812 00:11:27.848395 17730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:27.850204 17730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:27.850817 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17600
I20260812 00:11:27.850944 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-1/wal/instance
I20260812 00:11:27.851135 7511 external_mini_cluster.cc:1428] Setting key 50df4a0755db04a0fc727439c073099d
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.252@KRBTEST.COM; defaulting to no policy
I20260812 00:11:27.862036 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.851237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.7.85.254:40841)
0812 00:11:27.851320 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:27.851322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:27.851559 (+ 237us) server_negotiation.cc:407] Connection header received
0812 00:11:27.852350 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:27.852351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:27.852365 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:27.852429 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:27.853404 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.854190 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.855244 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.855600 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.855701 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.858574 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:27.858577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:27.858577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:27.858587 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:27.860378 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.860878 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.860879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.860880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.860950 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.861349 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.861350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.861351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.861475 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:27.861535 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:27.861885 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 00:11:27.861967 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 00:11:27.862215 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.851105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40841 (local address 127.0.0.1:47926)
0812 00:11:27.851395 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.851443 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:27.852218 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:27.852511 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:27.852516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:27.852950 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:27.853300 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.853306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.854305 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.854306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.855146 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.855149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.855744 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.855745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.855883 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.856575 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:27.856585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:27.858452 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:27.860488 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.860497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.860499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.860784 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.861054 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.861055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.861056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.861262 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.861554 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:27.861561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:27.861695 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 00:11:27.861794 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:11:27.861950 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":105,"threads_started":1}
I20260812 00:11:27.863158 17730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.252@KRBTEST.COM" created.
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:11:27.873159 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.863506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36362)
0812 00:11:27.863618 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.863629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:27.863678 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:27.864051 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:27.864052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:27.864194 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:27.864251 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.864253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.866017 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.866018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.866594 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.866596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.867303 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.867304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.867385 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.868170 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:27.868178 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:27.870012 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:27.872020 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.872021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.872021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.872204 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.872426 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.872427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.872427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.872486 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.872794 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:27.872795 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:27.872836 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:11:27.872902 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 00:11:27.873019 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 00:11:27.873217 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.863591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.7.85.253:41255)
0812 00:11:27.863840 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:27.863867 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:27.863890 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:11:27.863911 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:27.863913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:27.863925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:27.863998 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:27.864545 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.865885 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.866682 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.867192 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.867274 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.870128 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:27.870139 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:27.870141 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:27.870154 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:27.871913 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.872296 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.872302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.872302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.872359 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.872557 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.872557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.872558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.872722 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:27.872819 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:27.872940 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 00:11:27.873075 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
I20260812 00:11:27.874049 17730 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:27.874511 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.874370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38114)
0812 00:11:27.874427 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.874477 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 00:11:27.874646 17634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
I20260812 00:11:27.875088 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.874992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38116)
0812 00:11:27.875039 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.875064 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 00:11:27.875209 17730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:38951: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
I20260812 00:11:27.881790 17584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab.
Entry for principal kudu/127.7.85.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab.
I20260812 00:11:27.885563 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:38951
--webserver_interface=127.7.85.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab
--principal=kudu/127.7.85.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:40841,127.7.85.253:41255,127.7.85.252:38951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:11:27.890743 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.882299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.7.85.253:41255)
0812 00:11:27.882426 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:27.882429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:27.882437 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:11:27.882464 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:27.882465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:27.882474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:27.882550 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:27.882893 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.883466 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.884187 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.884427 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.884509 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.886830 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:27.886832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:27.886833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:27.886844 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:27.888919 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.889718 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.889719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.889720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.889801 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:27.890130 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:27.890132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:27.890132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:27.890269 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:27.890363 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:27.890535 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 00:11:27.890638 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 00:11:27.890743 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.882152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36366)
0812 00:11:27.882262 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.882284 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:27.882363 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:27.882556 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:27.882557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:27.882738 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:27.882809 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.882812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.883561 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.883562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.884090 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:27.884092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:27.884506 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:27.884507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:27.884596 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:27.885084 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:27.885097 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:27.886694 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:27.889118 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.889121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.889122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.889550 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.889947 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:27.889949 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:27.889950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:27.890047 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:27.890354 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:27.890355 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:27.890415 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 00:11:27.890507 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 00:11:27.890637 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 00:11:27.891429 17584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:27.891880 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.891732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38124)
0812 00:11:27.891814 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.891850 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 00:11:27.892309 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.892205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38134)
0812 00:11:27.892256 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.892284 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 00:11:27.892441 17584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:38951: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
I20260812 00:11:27.923657 17584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } attempt: 1
I20260812 00:11:27.924233 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.924073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38150)
0812 00:11:27.924140 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.924181 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:11:27.924417 17730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } attempt: 1
I20260812 00:11:27.924742 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.924621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38156)
0812 00:11:27.924681 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.924715 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 00:11:27.924839 17584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:38951: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
I20260812 00:11:27.925101 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.924705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38158)
0812 00:11:27.925022 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.925062 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292}
I20260812 00:11:27.925585 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.925482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38160)
0812 00:11:27.925529 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.925562 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 00:11:27.925716 17730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:38951: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
W20260812 00:11:27.981804 17746 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:11:27.982052 17746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:27.982070 17746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:27.982095 17746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:27.984774 17746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:27.984863 17746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:27.984880 17746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:27.984901 17746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:27.984915 17746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:27.987222 17746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:40841,127.7.85.253:41255,127.7.85.252:38951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab
--principal=kudu/127.7.85.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:38951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.85.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:27.987628 17746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:27.987900 17746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:27.993305 17753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:27.993307 17756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:27.993309 17752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:27.993640 17746 server_base.cc:1034] running on GCE node
I20260812 00:11:27.993860 17730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } attempt: 2
I20260812 00:11:27.993929 17746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:27.994350 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.994166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38172)
0812 00:11:27.994232 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.994266 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:11:27.994352 17746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:27.995009 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:27.994859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38176)
0812 00:11:27.994908 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:27.994935 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 00:11:27.995154 17730 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:38951: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
I20260812 00:11:27.995520 17746 hybrid_clock.cc:648] HybridClock initialized: now 1786493487995487 us; error 38 us; skew 500 ppm
Aug 12 00:11:27 dist-test-slave-z8x9 krb5kdc[17429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:27.998651 17746 init.cc:377] Logged in from keytab as kudu/127.7.85.252@KRBTEST.COM (short username kudu)
I20260812 00:11:27.999238 17746 webserver.cc:460] Webserver started at http://127.7.85.252:39759/ using document root <none> and password file <none>
I20260812 00:11:27.999552 17746 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:27.999629 17746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:27.999796 17746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:27.999991 17584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } attempt: 2
I20260812 00:11:28.000447 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.000318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38186)
0812 00:11:28.000379 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.000413 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:11:28.000789 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.000689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38198)
0812 00:11:28.000740 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.000766 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 00:11:28.000885 17584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:38951: Network error: Client connection negotiation failed: client connection to 127.7.85.252:38951: connect: Connection refused (error 111)
I20260812 00:11:28.001116 17746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data/instance:
uuid: "871394a10bd7479a9073bab65771dd9d"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "2833761a975b7fbe4948f12e87c254e7"
server_key_iv: "702cdb6c488ba23008624e2e4fc8caf8"
server_key_version: "encryptionkey@0"
I20260812 00:11:28.001554 17746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal/instance:
uuid: "871394a10bd7479a9073bab65771dd9d"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "2833761a975b7fbe4948f12e87c254e7"
server_key_iv: "702cdb6c488ba23008624e2e4fc8caf8"
server_key_version: "encryptionkey@0"
I20260812 00:11:28.003036 17746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 00:11:28.003948 17768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:28.004197 17746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:28.004319 17746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal
uuid: "871394a10bd7479a9073bab65771dd9d"
format_stamp: "Formatted at 2026-08-12 00:11:27 on dist-test-slave-z8x9"
server_key: "2833761a975b7fbe4948f12e87c254e7"
server_key_iv: "702cdb6c488ba23008624e2e4fc8caf8"
server_key_version: "encryptionkey@0"
I20260812 00:11:28.004431 17746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:28.022938 17746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:28.025570 17746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:28.025763 17746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:28.031105 17746 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.252:38951
I20260812 00:11:28.031144 17861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.252:38951 every 8 connection(s)
I20260812 00:11:28.031528 17746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
I20260812 00:11:28.032616 17864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:28.034336 17864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:28.035344 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 17746
I20260812 00:11:28.035465 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786493185917310-7511-0/minicluster-data/master-2/wal/instance
I20260812 00:11:28.035679 7511 external_mini_cluster.cc:1428] Setting key 02195c30bd7155946362db04ade87ecd
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:11:28.046293 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.035164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47936 (local address 127.7.85.254:40841)
0812 00:11:28.035248 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.035250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.035454 (+ 204us) server_negotiation.cc:407] Connection header received
0812 00:11:28.036478 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.036480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.036493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.036555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.037343 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.037910 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.039271 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.039588 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.039656 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.042634 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.042635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.042636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.042652 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.044728 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.045160 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.045161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.045162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.045252 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.045499 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.045500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.045501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.045630 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.045810 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.046036 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.046139 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 00:11:28.046499 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.036233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.7.85.252:38951)
0812 00:11:28.036442 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.036447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.036487 (+ 40us) server_negotiation.cc:407] Connection header received
0812 00:11:28.036599 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.036601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.036620 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.036723 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.037557 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.038716 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.039671 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.040144 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.040204 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.042980 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.042989 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.042990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.042998 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.044686 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.045215 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.045219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.045219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.045317 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.045546 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.045547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.045547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.045793 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.045968 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.046041 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.046183 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
I20260812 00:11:28.046496 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.035079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:40841 (local address 127.0.0.1:47936)
0812 00:11:28.035304 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.035359 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:28.036339 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:28.036605 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:28.036610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:28.036957 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:28.037117 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.037122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.038002 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.038003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.039159 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.039162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.039709 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.039711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.039856 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.040572 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:28.040581 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:28.042373 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:28.044848 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.044857 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.044860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.045075 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.045323 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.045324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.045325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.045427 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.045782 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:28.045793 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:28.045877 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 00:11:28.046009 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 00:11:28.046273 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":253,"thread_start_us":110,"threads_started":1}
I20260812 00:11:28.047142 17864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:11:28.056372 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.047487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.7.85.253:41255)
0812 00:11:28.047531 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.047533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.047544 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:11:28.047600 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.047601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.047613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.047674 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.048230 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.049085 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.050179 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.050467 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.050548 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.052739 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.052741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.052742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.052750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.055082 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.055524 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.055525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.055526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.055592 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.055813 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.055814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.055815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.055942 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.056017 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.056189 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.056264 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:11:28.056372 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.047428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:41255 (local address 127.0.0.1:36376)
0812 00:11:28.047468 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.047481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:28.047529 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:28.047683 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:28.047684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:28.048004 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:28.048065 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.048068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.049289 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.049290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.050062 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.050065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.050594 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.050595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.050655 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.051151 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:28.051156 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:28.052641 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:28.055196 (+ 2555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.055197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.055197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.055407 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.055672 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.055673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.055673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.055737 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.056022 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:28.056024 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:28.056077 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 00:11:28.056147 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 00:11:28.056260 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 00:11:28.056473 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.047667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.7.85.254:40841)
0812 00:11:28.047703 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.047705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.047941 (+ 236us) server_negotiation.cc:407] Connection header received
0812 00:11:28.047963 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.047964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.047973 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.048019 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.048823 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.049475 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.050382 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.050671 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.050724 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.053568 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.053569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.053570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.053579 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.055289 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.055636 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.055637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.055638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.055703 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.055922 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.055923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.055924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.056044 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.056215 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.056280 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.056367 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 00:11:28.057396 17864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:11:28.065384 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.057737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38220)
0812 00:11:28.057770 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.057780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:28.057811 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:28.057936 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:28.057937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:28.058077 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:28.058110 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.058112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.058795 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.058796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.059290 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.059291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.059867 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.059868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.059931 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.060488 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:28.060493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:28.061827 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:28.064095 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.064096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.064097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.064273 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.064562 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.064563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.064564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.064628 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.064947 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:28.064948 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:28.064989 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:11:28.065073 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:11:28.065248 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 00:11:28.065448 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.057809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38220 (local address 127.7.85.252:38951)
0812 00:11:28.057846 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.057848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.057854 (+ 6us) server_negotiation.cc:407] Connection header received
0812 00:11:28.057872 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.057873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.057881 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.057933 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.058199 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.058699 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.059369 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.059598 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.059867 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.061939 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.061941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.061941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.061948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.063989 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.064385 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.064387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.064388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.064456 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.064734 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.064735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.064735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.064862 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.064944 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.065112 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.065301 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":22}
I20260812 00:11:28.066677 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.057347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.7.85.253:41255)
0812 00:11:28.057614 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.057616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.057625 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:28.057646 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.057646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.057656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.057711 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.058266 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.058751 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.059427 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.059638 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.059714 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.062585 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.062586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.062587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.062596 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.064594 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.064952 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.064954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.064954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.065020 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.065296 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.065297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.065298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.065420 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.066248 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.066481 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.066575 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260812 00:11:28.067301 17864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: Bootstrap starting.
I20260812 00:11:28.068347 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:28.068361 17864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:28.068662 17864 log.cc:824] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:28.069578 17864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: No bootstrap required, opened a new log
I20260812 00:11:28.069404 17531 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4ef67f42a8e345899492ce76c4a41ab5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47938
I20260812 00:11:28.071847 17864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.072028 17864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871394a10bd7479a9073bab65771dd9d, State: Initialized, Role: FOLLOWER
I20260812 00:11:28.072111 17864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.072487 17864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:28.072440 17880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.072571 17880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:11:28.074517 17898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:28.074621 17898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:28.074934 17746 master_runner.cc:186] Error getting master registration for 127.7.85.254:40841: OK, instance_id { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" instance_seqno: 1786493487649907 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:11:28.077374 17746 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:11:28.086371 17730 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } attempt: 3
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:11:28.093777 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.086877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.7.85.252:38951)
0812 00:11:28.086922 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.086924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.086935 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:11:28.087021 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.087022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.087033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.087093 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.087449 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.087944 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.088628 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.088832 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.088920 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.090921 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.090922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.090923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.090931 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.092470 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.092829 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.092830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.092830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.092896 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.093107 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.093107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.093108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.093334 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.093419 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.093578 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.093674 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:11:28.093777 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.086787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38232)
0812 00:11:28.086859 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.086874 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:28.086948 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:28.087134 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:28.087135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:28.087317 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:28.087363 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.087365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.088045 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.088046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.088540 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.088541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.088923 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.088924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.088979 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.089529 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:28.089535 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:28.090814 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:28.092566 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.092568 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.092568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.092754 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.092971 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.092972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.092972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.093037 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.093409 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:28.093410 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:28.093461 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 00:11:28.093552 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 00:11:28.093669 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 00:11:28.095395 17730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: Bootstrap starting.
I20260812 00:11:28.096091 17730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:28.096452 17730 log.cc:824] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:28.097441 17730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: No bootstrap required, opened a new log
I20260812 00:11:28.100049 17584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } attempt: 3
I20260812 00:11:28.100294 17730 raft_consensus.cc:348] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.100798 17730 raft_consensus.cc:729] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77012262040496395638848b3e09da7, State: Initialized, Role: FOLLOWER
I20260812 00:11:28.100927 17730 consensus_queue.cc:260] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.101439 17730 sys_catalog.cc:564] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 00:11:28.103698 17922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:28.103758 17922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:28.106436 17902 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.106602 17902 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:28.103730 17600 master_runner.cc:186] Error getting master registration for 127.7.85.254:40841: OK, instance_id { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" instance_seqno: 1786493487649907 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:11:28.107493 17600 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 00:11:28 dist-test-slave-z8x9 krb5kdc[17429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:11:28.114104 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.100524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.7.85.252:38951)
0812 00:11:28.100574 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.100576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.100589 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:11:28.100617 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.100618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.100635 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.100741 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.101252 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.101985 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.106755 (+ 4770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.107082 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.107179 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.110230 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.110232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.110233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.110241 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.112327 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.112970 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.112972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.112972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.113045 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.113288 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.113289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.113290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.113445 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.113512 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.113876 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.113961 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:11:28.114593 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.100434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:38951 (local address 127.0.0.1:38246)
0812 00:11:28.100498 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:28.100509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:28.100553 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:28.100753 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:28.100755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:28.100970 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:28.101029 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.101032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.102152 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.102153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.103000 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:28.103003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.107211 (+ 4208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.107224 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:28.107316 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.108204 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:28.108211 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:28.110085 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:28.112468 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.112469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.112469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.112834 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.113123 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:28.113124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:28.113124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:28.113212 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:28.113541 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:28.113542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:28.113594 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 00:11:28.113658 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 00:11:28.114489 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:11:28.116668 17584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: Bootstrap starting.
I20260812 00:11:28.117622 17584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:28.117944 17584 log.cc:824] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:28.118980 17584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: No bootstrap required, opened a new log
I20260812 00:11:28.120584 17584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.120814 17584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef67f42a8e345899492ce76c4a41ab5, State: Initialized, Role: FOLLOWER
I20260812 00:11:28.120972 17584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.121410 17584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:28.121475 17531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 00:11:28.121403 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.121577 17531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:28.121579 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:28.122788 17531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.123085 17531 leader_election.cc:288] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [CANDIDATE]: Term 1 election: Requested vote from peers d77012262040496395638848b3e09da7 (127.7.85.253:41255), 871394a10bd7479a9073bab65771dd9d (127.7.85.252:38951)
I20260812 00:11:28.124953 17684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ef67f42a8e345899492ce76c4a41ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d77012262040496395638848b3e09da7"
I20260812 00:11:28.125129 17684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:28.126153 17684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef67f42a8e345899492ce76c4a41ab5 in term 1.
I20260812 00:11:28.126456 17483 leader_election.cc:302] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef67f42a8e345899492ce76c4a41ab5, d77012262040496395638848b3e09da7; no voters:
I20260812 00:11:28.126665 17928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:28.126884 17928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [term 1 LEADER]: Becoming Leader. State: Replica: 4ef67f42a8e345899492ce76c4a41ab5, State: Running, Role: LEADER
I20260812 00:11:28.126971 17814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ef67f42a8e345899492ce76c4a41ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "871394a10bd7479a9073bab65771dd9d"
I20260812 00:11:28.127105 17814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:28.127027 17928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } }
I20260812 00:11:28.127609 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef67f42a8e345899492ce76c4a41ab5. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.127743 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:28.128049 17814 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef67f42a8e345899492ce76c4a41ab5 in term 1.
I20260812 00:11:28.130764 17449 master_runner.cc:263] No leader master found from master 4ef67f42a8e345899492ce76c4a41ab5
I20260812 00:11:28.130823 17449 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 00:11:28.131572 17956 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:28.131642 17956 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:28.137473 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.128782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.7.85.252:38951)
0812 00:11:28.128854 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.128856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.129099 (+ 243us) server_negotiation.cc:407] Connection header received
0812 00:11:28.129246 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.129247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.129260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.129292 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.130149 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.130901 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.132163 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.132492 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.132571 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.134331 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.134333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.134333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.134341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.136099 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.136536 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.136538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.136538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.136596 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.136804 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.136805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.136805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.136937 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.137010 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.137284 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.137369 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 00:11:28.141255 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.129362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.7.85.254:40841)
0812 00:11:28.129404 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.129406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.133353 (+ 3947us) server_negotiation.cc:407] Connection header received
0812 00:11:28.133430 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.133431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.133445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.133498 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.134299 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.135024 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.136118 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.136427 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.136470 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.137961 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.137962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.137962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.137970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.139934 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.140356 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.140357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.140358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.140418 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.140617 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.140618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.140618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.140749 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.140822 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.141026 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.141107 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 00:11:28.143561 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:28.129694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.7.85.253:41255)
0812 00:11:28.129738 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.129739 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.133539 (+ 3800us) server_negotiation.cc:407] Connection header received
0812 00:11:28.133638 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.133640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.133651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.133698 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.134380 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.135085 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.138094 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:28.138416 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:28.138462 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:28.140047 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:28.140049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:28.140049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:28.140057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:28.141817 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.142728 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.142729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.142730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.142785 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:28.142974 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:28.142975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:28.142975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:28.143098 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:28.143166 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:28.143388 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 00:11:28.143466 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:11:28.146134 17964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:28.146368 17964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 00:11:28.147616 17492 tablet.cc:2369] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:11:28.154991 17684 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [term 1 FOLLOWER]: Refusing update from remote peer 4ef67f42a8e345899492ce76c4a41ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:11:28.155300 17814 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [term 1 FOLLOWER]: Refusing update from remote peer 4ef67f42a8e345899492ce76c4a41ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:11:28.155642 17962 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:11:28.155781 17928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:11:28.157333 17902 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef67f42a8e345899492ce76c4a41ab5. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.157461 17902 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:28.158399 17880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef67f42a8e345899492ce76c4a41ab5. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.158492 17880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:28.159600 17962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.159713 17962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:28.159929 17970 mvcc.cc:205] Tried to move back new op lower bound from 7317477327479173120 to 7317477327369814016. Current Snapshot: MvccSnapshot[applied={T|T < 7317477327479173120}]
I20260812 00:11:28.160167 17902 sys_catalog.cc:455] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.160259 17902 sys_catalog.cc:458] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:28.160501 17973 mvcc.cc:205] Tried to move back new op lower bound from 7317477327479173120 to 7317477327369814016. Current Snapshot: MvccSnapshot[applied={T|T < 7317477327479173120}]
I20260812 00:11:28.161089 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.161202 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:28.161527 17880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ef67f42a8e345899492ce76c4a41ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef67f42a8e345899492ce76c4a41ab5" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 40841 } } peers { permanent_uuid: "d77012262040496395638848b3e09da7" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 41255 } } peers { permanent_uuid: "871394a10bd7479a9073bab65771dd9d" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 38951 } } }
I20260812 00:11:28.161614 17880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:28.164108 17972 mvcc.cc:205] Tried to move back new op lower bound from 7317477327479173120 to 7317477327369814016. Current Snapshot: MvccSnapshot[applied={T|T < 7317477327479173120}]
I20260812 00:11:28.164773 17964 catalog_manager.cc:1347] Generated new cluster ID: ad4598d33f3940359bd6b7a849bd6690
I20260812 00:11:28.164832 17964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:28.182668 17964 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:28.183614 17964 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:28.203809 17964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ef67f42a8e345899492ce76c4a41ab5: Generated new TSK 0
I20260812 00:11:28.204053 17964 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:11:28.236693 17591 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.235179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.7.85.254:40841)
0812 00:11:28.235232 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.235235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.235912 (+ 677us) server_negotiation.cc:407] Connection header received
0812 00:11:28.236037 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.236038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.236051 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.236083 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.236606 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47954: BlockingRecv error: recv got EOF from 127.0.0.1:47954 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 00:11:28.238376 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.235589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.7.85.253:41255)
0812 00:11:28.235633 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.235635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.237741 (+ 2106us) server_negotiation.cc:407] Connection header received
0812 00:11:28.237870 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.237871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.237883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.237916 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.238286 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36398: BlockingRecv error: recv got EOF from 127.0.0.1:36398 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 00:11:28.239418 17876 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.237384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.7.85.252:38951)
0812 00:11:28.237431 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.237433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.238778 (+ 1345us) server_negotiation.cc:407] Connection header received
0812 00:11:28.238901 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.238902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.238913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.238945 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.239352 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38252: BlockingRecv error: recv got EOF from 127.0.0.1:38252 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 00:11:28.242417 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.241089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36404 (local address 127.7.85.253:41255)
0812 00:11:28.241131 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.241133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.241831 (+ 698us) server_negotiation.cc:407] Connection header received
0812 00:11:28.241950 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.241951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.241973 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.242006 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.242361 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36404: BlockingRecv error: recv got EOF from 127.0.0.1:36404 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 00:11:28.246464 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.245412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.7.85.253:41255)
0812 00:11:28.245454 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.245456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.245808 (+ 352us) server_negotiation.cc:407] Connection header received
0812 00:11:28.245939 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.245940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.245953 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.245986 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.246407 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36406: BlockingRecv error: recv got EOF from 127.0.0.1:36406 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 00:11:28.251068 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.250009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.7.85.253:41255)
0812 00:11:28.250051 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.250053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.250342 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:11:28.250446 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.250447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.250458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.250488 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.251013 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36412: BlockingRecv error: recv got EOF from 127.0.0.1:36412 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 00:11:28.257582 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.256509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.7.85.253:41255)
0812 00:11:28.256552 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.256554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.256855 (+ 301us) server_negotiation.cc:407] Connection header received
0812 00:11:28.256961 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.256962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.256974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.257005 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.257521 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36416: BlockingRecv error: recv got EOF from 127.0.0.1:36416 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 00:11:28.268213 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.267061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.7.85.253:41255)
0812 00:11:28.267108 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.267111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.267526 (+ 415us) server_negotiation.cc:407] Connection header received
0812 00:11:28.267655 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.267657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.267670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.267702 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.268147 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36426: BlockingRecv error: recv got EOF from 127.0.0.1:36426 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 00:11:28.286839 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.285806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.7.85.253:41255)
0812 00:11:28.285855 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.285857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.286170 (+ 313us) server_negotiation.cc:407] Connection header received
0812 00:11:28.286285 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.286286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.286299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.286330 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.286773 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36430: BlockingRecv error: recv got EOF from 127.0.0.1:36430 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 00:11:28.321336 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.320563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36442 (local address 127.7.85.253:41255)
0812 00:11:28.320628 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.320631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.320749 (+ 118us) server_negotiation.cc:407] Connection header received
0812 00:11:28.320784 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.320786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.320799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.320862 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.321268 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36442: BlockingRecv error: recv got EOF from 127.0.0.1:36442 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 00:11:28.387355 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.386644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.7.85.253:41255)
0812 00:11:28.386702 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.386704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.386763 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:11:28.386818 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.386819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.386830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.386883 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.387272 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36450: BlockingRecv error: recv got EOF from 127.0.0.1:36450 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 00:11:28.517621 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.516680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.7.85.253:41255)
0812 00:11:28.516793 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.516795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.516808 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:11:28.516864 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.516865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.516876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.516935 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.517530 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36452: BlockingRecv error: recv got EOF from 127.0.0.1:36452 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260812 00:11:28.775606 17742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:28.774985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.7.85.253:41255)
0812 00:11:28.775065 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:28.775067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:28.775076 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:28.775103 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:28.775104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:28.775114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:28.775186 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:28.775522 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36468: BlockingRecv error: recv got EOF from 127.0.0.1:36468 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:11:29.075169 17898 catalog_manager.cc:1259] Loaded cluster ID: ad4598d33f3940359bd6b7a849bd6690
I20260812 00:11:29.075258 17898 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: loading cluster ID for follower catalog manager: success
I20260812 00:11:29.076102 17898 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: acquiring CA information for follower catalog manager: success
I20260812 00:11:29.076443 17898 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 871394a10bd7479a9073bab65771dd9d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:11:29.104250 17922 catalog_manager.cc:1259] Loaded cluster ID: ad4598d33f3940359bd6b7a849bd6690
I20260812 00:11:29.104367 17922 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: loading cluster ID for follower catalog manager: success
I20260812 00:11:29.105155 17922 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: acquiring CA information for follower catalog manager: success
I20260812 00:11:29.105688 17922 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d77012262040496395638848b3e09da7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:11:29.291668 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:29.288927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36474 (local address 127.7.85.253:41255)
0812 00:11:29.289039 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:29.289041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:29.289050 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:29.289079 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:29.289080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:29.289132 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:29.289286 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:29.289523 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.290046 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.290855 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.291078 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.291149 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:29.291324 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:29.291446 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:29.291522 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:29.291567 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 00:11:29.291586 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":63,"threads_started":1}
I20260812 00:11:29.296687 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:29.294066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.7.85.252:38951)
0812 00:11:29.294214 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:29.294216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:29.294225 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:29.294262 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:29.294263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:29.294301 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:29.294377 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:29.294464 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.294940 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.295697 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.295916 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.296000 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:29.296162 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:29.296276 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:29.296355 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:29.296558 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 00:11:29.296592 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260812 00:11:29.302110 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:29.298913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.7.85.253:41255)
0812 00:11:29.298963 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:29.298966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:29.299256 (+ 290us) server_negotiation.cc:407] Connection header received
0812 00:11:29.299310 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:29.299311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:29.299323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:29.299374 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:29.299559 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.300278 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.301098 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.301394 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.301475 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:29.301635 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:29.301674 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:29.301743 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:29.301919 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 00:11:29.301948 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:11:29.302229 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:29.298839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38274 (local address 127.7.85.252:38951)
0812 00:11:29.298884 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:29.298886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:29.299036 (+ 150us) server_negotiation.cc:407] Connection header received
0812 00:11:29.299176 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:29.299177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:29.299194 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:29.299258 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:29.299455 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.300231 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.301259 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.301599 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.301657 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:29.301836 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:29.301878 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:29.301931 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:29.302051 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 00:11:29.302074 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:11:29.302305 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:29.298795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47958 (local address 127.7.85.254:40841)
0812 00:11:29.298948 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:29.298949 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:29.299068 (+ 119us) server_negotiation.cc:407] Connection header received
0812 00:11:29.299172 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:29.299173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:29.299213 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:29.299320 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:29.299435 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.299988 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.301129 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:29.301471 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:29.301513 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:29.301789 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:29.301915 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:29.301975 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:29.302185 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 00:11:29.302216 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260812 00:11:29.308620 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17449
I20260812 00:11:29.331951 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17600
I20260812 00:11:29.350099 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 17746
2026-08-12T00:11:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3328 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:11:29 dist-test-slave-z8x9 krb5kdc[18129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:11:29 dist-test-slave-z8x9 krb5kdc[18129](info): set up 2 sockets
Aug 12 00:11:29 dist-test-slave-z8x9 krb5kdc[18129](info): commencing operation
krb5kdc: starting...
W20260812 00:11:30.728997 7511 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/test-user.keytab.
Aug 12 00:11:30 dist-test-slave-z8x9 krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.85.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
Entry for principal kudu/127.7.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab.
I20260812 00:11:30.873466 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36965
--webserver_interface=127.7.85.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36965,127.7.85.253:44689,127.7.85.252:46359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:11:30.965090 18147 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:11:30.965371 18147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:30.965390 18147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:30.965413 18147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:30.966629 18147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:30.966683 18147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:30.966698 18147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:30.966711 18147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:30.966724 18147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:30.968771 18147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36965,127.7.85.253:44689,127.7.85.252:46359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.254.keytab
--principal=kudu/127.7.85.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.254:36965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.85.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:30.969172 18147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:30.969513 18147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:30.975118 18158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:30.975178 18165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:30.975262 18147 server_base.cc:1034] running on GCE node
W20260812 00:11:30.975226 18159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:30.975695 18147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:30.976033 18147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:30.977280 18147 hybrid_clock.cc:648] HybridClock initialized: now 1786493490977252 us; error 53 us; skew 500 ppm
Aug 12 00:11:30 dist-test-slave-z8x9 krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:30.980157 18147 init.cc:377] Logged in from keytab as kudu/127.7.85.254@KRBTEST.COM (short username kudu)
I20260812 00:11:30.980757 18147 webserver.cc:460] Webserver started at http://127.7.85.254:33289/ using document root <none> and password file <none>
I20260812 00:11:30.981020 18147 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:30.981098 18147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:30.981300 18147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:30.982529 18147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data/instance:
uuid: "3e77e1fe51974235984011422032020b"
format_stamp: "Formatted at 2026-08-12 00:11:30 on dist-test-slave-z8x9"
server_key: "51778242695b77bb862b50c0ec1dea94"
server_key_iv: "1b5428ddad4babbebe7ef32413ef35d2"
server_key_version: "encryptionkey@0"
I20260812 00:11:30.983065 18147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal/instance:
uuid: "3e77e1fe51974235984011422032020b"
format_stamp: "Formatted at 2026-08-12 00:11:30 on dist-test-slave-z8x9"
server_key: "51778242695b77bb862b50c0ec1dea94"
server_key_iv: "1b5428ddad4babbebe7ef32413ef35d2"
server_key_version: "encryptionkey@0"
I20260812 00:11:30.985039 18147 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:11:30.986132 18176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:30.986613 18147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:30.986763 18147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal
uuid: "3e77e1fe51974235984011422032020b"
format_stamp: "Formatted at 2026-08-12 00:11:30 on dist-test-slave-z8x9"
server_key: "51778242695b77bb862b50c0ec1dea94"
server_key_iv: "1b5428ddad4babbebe7ef32413ef35d2"
server_key_version: "encryptionkey@0"
I20260812 00:11:30.986857 18147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:31.003345 18147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:31.007035 18147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:31.007305 18147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:31.012926 18147 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.254:36965
I20260812 00:11:31.013037 18281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.254:36965 every 8 connection(s)
I20260812 00:11:31.013468 18147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/data/info.pb
I20260812 00:11:31.014876 18282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:31.016916 18282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.020553 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18147
I20260812 00:11:31.020785 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-0/wal/instance
I20260812 00:11:31.021648 7511 external_mini_cluster.cc:1428] Setting key 7b5da86843715d91ac017aeac637c0be
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:11:31.030393 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.017747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36965 (local address 127.0.0.1:48374)
0812 00:11:31.017952 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.018000 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.018490 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.018792 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.018794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.019093 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.019260 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.019264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.020344 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.020345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.021328 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.021332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.022396 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.022398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.022480 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.023166 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.023176 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.025126 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.027831 (+ 2705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.027843 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.027847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.028212 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.028558 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.028559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.028559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.028800 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.029397 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.029406 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.029505 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.029633 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.029873 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260812 00:11:31.030396 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.017871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.7.85.254:36965)
0812 00:11:31.018054 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.018059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.018101 (+ 42us) server_negotiation.cc:407] Connection header received
0812 00:11:31.018649 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.018651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.018670 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.018763 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.019364 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.020219 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.021483 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.022205 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.022402 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.025328 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.025341 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.025341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.025353 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.027660 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.028347 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.028352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.028352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.028466 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.028917 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.028918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.028918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.029311 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.029395 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.029728 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.030017 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260812 00:11:31.031538 18282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.031999 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.031837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59260)
0812 00:11:31.031901 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.031965 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 00:11:31.032202 18180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
I20260812 00:11:31.032755 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.032639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59268)
0812 00:11:31.032694 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.032729 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
WARNING: no policy specified for kudu/127.7.85.253@KRBTEST.COM; defaulting to no policy
W20260812 00:11:31.032884 18282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:44689: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab.
Entry for principal kudu/127.7.85.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab.
I20260812 00:11:31.058187 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:44689
--webserver_interface=127.7.85.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab
--principal=kudu/127.7.85.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36965,127.7.85.253:44689,127.7.85.252:46359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:11:31.082140 18282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } attempt: 1
I20260812 00:11:31.082760 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.082594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59280)
0812 00:11:31.082662 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.082723 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 00:11:31.083235 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.083112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59292)
0812 00:11:31.083168 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.083202 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 00:11:31.083374 18282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:44689: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
W20260812 00:11:31.146552 18293 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:11:31.146814 18293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:31.146832 18293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:31.146855 18293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:31.147965 18293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:31.148007 18293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:31.148021 18293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:31.148034 18293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:31.148046 18293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:31.149588 18293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36965,127.7.85.253:44689,127.7.85.252:46359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.253.keytab
--principal=kudu/127.7.85.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.253:44689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.85.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:31.149821 18293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:31.150007 18293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:11:31.151618 18282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } attempt: 2
I20260812 00:11:31.152495 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.152281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59294)
0812 00:11:31.152399 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.152453 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 00:11:31.153234 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.152971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59302)
0812 00:11:31.153074 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.153127 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 00:11:31.153398 18282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.253:44689: Network error: Client connection negotiation failed: client connection to 127.7.85.253:44689: connect: Connection refused (error 111)
W20260812 00:11:31.155871 18313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:31.155857 18308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:31.155861 18301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:31.156388 18293 server_base.cc:1034] running on GCE node
I20260812 00:11:31.156960 18293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:31.157403 18293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:31.158658 18293 hybrid_clock.cc:648] HybridClock initialized: now 1786493491158634 us; error 32 us; skew 500 ppm
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:31.162433 18293 init.cc:377] Logged in from keytab as kudu/127.7.85.253@KRBTEST.COM (short username kudu)
I20260812 00:11:31.163340 18293 webserver.cc:460] Webserver started at http://127.7.85.253:43987/ using document root <none> and password file <none>
I20260812 00:11:31.163686 18293 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:31.163769 18293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:31.163957 18293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:31.165405 18293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data/instance:
uuid: "a4fed0fa93844b148148285a7b4ef3b0"
format_stamp: "Formatted at 2026-08-12 00:11:31 on dist-test-slave-z8x9"
server_key: "9e77bab017b18986327da4685b27bc61"
server_key_iv: "37d40f24bdf79de062a1356d4dd0a164"
server_key_version: "encryptionkey@0"
I20260812 00:11:31.165845 18293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal/instance:
uuid: "a4fed0fa93844b148148285a7b4ef3b0"
format_stamp: "Formatted at 2026-08-12 00:11:31 on dist-test-slave-z8x9"
server_key: "9e77bab017b18986327da4685b27bc61"
server_key_iv: "37d40f24bdf79de062a1356d4dd0a164"
server_key_version: "encryptionkey@0"
I20260812 00:11:31.167725 18293 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:11:31.168952 18320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:31.169263 18293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:31.169394 18293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal
uuid: "a4fed0fa93844b148148285a7b4ef3b0"
format_stamp: "Formatted at 2026-08-12 00:11:31 on dist-test-slave-z8x9"
server_key: "9e77bab017b18986327da4685b27bc61"
server_key_iv: "37d40f24bdf79de062a1356d4dd0a164"
server_key_version: "encryptionkey@0"
I20260812 00:11:31.169512 18293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:31.181504 18293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:31.185503 18293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:31.185818 18293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:31.191582 18293 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.253:44689
I20260812 00:11:31.191730 18419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.253:44689 every 8 connection(s)
I20260812 00:11:31.192091 18293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/data/info.pb
I20260812 00:11:31.193326 18423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:31.195173 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18293
I20260812 00:11:31.195288 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-1/wal/instance
I20260812 00:11:31.195127 18423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.195524 7511 external_mini_cluster.cc:1428] Setting key b45d909a3d9ba3ac18578e42710d964b
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.85.252@KRBTEST.COM; defaulting to no policy
I20260812 00:11:31.207049 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.196138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.7.85.254:36965)
0812 00:11:31.196231 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.196233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.196522 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:11:31.197379 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.197381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.197392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.197454 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.198440 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.198967 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.200089 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.200464 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.200561 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.203244 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.203247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.203247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.203258 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.205203 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.205789 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.205790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.205791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.205855 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.206182 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.206184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.206184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.206356 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.206418 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.206887 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.206972 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 00:11:31.207357 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.196115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36965 (local address 127.0.0.1:48380)
0812 00:11:31.196362 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.196415 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.197242 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.197541 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.197546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.197991 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.198315 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.198321 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.199076 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.199077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.199975 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.199977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.200602 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.200604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.200766 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.201565 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.201575 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.203122 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.205357 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.205367 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.205370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.205706 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.205953 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.205954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.205955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.206087 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.206480 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.206488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.206638 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.206816 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.207060 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":127,"threads_started":1}
I20260812 00:11:31.208546 18423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.85.252@KRBTEST.COM" created.
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:11:31.217669 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.208917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59304)
0812 00:11:31.208993 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.209004 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.209046 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.209493 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.209494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.209666 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.209763 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.209765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.210637 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.210639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.211156 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.211157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.211815 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.211817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.211890 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.212507 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.212519 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.214169 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.216407 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.216409 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.216409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.216595 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.216842 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.216843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.216843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.216907 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.217308 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.217309 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.217354 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.217429 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.217547 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 00:11:31.217782 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.209008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.7.85.253:44689)
0812 00:11:31.209328 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.209333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.209352 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:11:31.209373 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.209375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.209387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.209484 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.209844 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.210541 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.211247 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.211709 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.211793 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.214266 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.214278 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.214279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.214288 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.216289 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.216681 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.216685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.216685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.216751 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.216971 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.216972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.216972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.217154 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.217334 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.217464 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.217648 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":291,"thread_start_us":258,"threads_started":1}
I20260812 00:11:31.219506 18423 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.219980 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.219815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57174)
0812 00:11:31.219901 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.219954 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 00:11:31.220273 18330 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
I20260812 00:11:31.221155 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.221001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57186)
0812 00:11:31.221092 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.221130 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 00:11:31.221400 18423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:46359: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.85.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab.
Entry for principal kudu/127.7.85.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab.
I20260812 00:11:31.234519 7511 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskaxXthI/build/release/bin/kudu
/tmp/dist-test-taskaxXthI/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:46359
--webserver_interface=127.7.85.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab
--principal=kudu/127.7.85.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.85.212:45259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.85.254:36965,127.7.85.253:44689,127.7.85.252:46359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:11:31.244640 18282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } attempt: 3
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:11:31.253391 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.245233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59320 (local address 127.7.85.253:44689)
0812 00:11:31.245311 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.245313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.245324 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:11:31.245397 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.245398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.245416 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.245482 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.245852 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.246371 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.247086 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.247330 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.247409 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.249926 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.249928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.249928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.249937 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.251877 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.252276 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.252277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.252278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.252347 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.252740 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.252741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.252741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.252899 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.252983 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.253150 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.253282 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 00:11:31.253391 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.245105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59320)
0812 00:11:31.245202 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.245215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.245289 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.245488 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.245489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.245688 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.245759 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.245761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.246460 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.246461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.246995 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.246996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.247408 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.247409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.247491 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.248143 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.248148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.249777 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.251995 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.251996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.251997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.252194 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.252426 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.252427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.252428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.252496 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.252975 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.252976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.253039 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.253124 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.253291 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 00:11:31.253917 18282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.254380 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.254220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57188)
0812 00:11:31.254298 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.254335 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 00:11:31.254761 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.254659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57192)
0812 00:11:31.254707 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.254732 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 00:11:31.254858 18282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:46359: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
I20260812 00:11:31.270740 18423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } attempt: 1
I20260812 00:11:31.271517 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.271364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57202)
0812 00:11:31.271430 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.271477 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 00:11:31.271898 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.271797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57212)
0812 00:11:31.271845 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.271873 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 00:11:31.272006 18423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:46359: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
I20260812 00:11:31.286079 18282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } attempt: 1
I20260812 00:11:31.286815 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.286639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57224)
0812 00:11:31.286725 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.286777 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 00:11:31.287534 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.287332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57236)
0812 00:11:31.287435 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.287485 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260812 00:11:31.287690 18282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:46359: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
W20260812 00:11:31.326892 18437 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:11:31.327128 18437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:11:31.327147 18437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:11:31.327168 18437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:11:31.328341 18437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:11:31.328387 18437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:11:31.328401 18437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:11:31.328414 18437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:11:31.328426 18437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:11:31.330018 18437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.85.212:45259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.85.254:36965,127.7.85.253:44689,127.7.85.252:46359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/krb5kdc/kudu_127.7.85.252.keytab
--principal=kudu/127.7.85.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.85.252:46359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.85.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 22:58:32 UTC on ccycloud.quasar-jwjgej.root.comops.site
build id 4581605
I20260812 00:11:31.330305 18437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:11:31.330560 18437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:11:31.335952 18449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:31.336009 18447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:11:31.335974 18451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:11:31.336007 18437 server_base.cc:1034] running on GCE node
I20260812 00:11:31.336398 18437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:11:31.336730 18437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:11:31.338037 18437 hybrid_clock.cc:648] HybridClock initialized: now 1786493491337999 us; error 28 us; skew 500 ppm
I20260812 00:11:31.340166 18423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } attempt: 2
I20260812 00:11:31.340620 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.340486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57244)
0812 00:11:31.340550 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.340583 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:11:31.340937 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.340846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57260)
0812 00:11:31.340890 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.340915 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 00:11:31.341046 18423 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:46359: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
I20260812 00:11:31.341373 18437 init.cc:377] Logged in from keytab as kudu/127.7.85.252@KRBTEST.COM (short username kudu)
I20260812 00:11:31.341895 18437 webserver.cc:460] Webserver started at http://127.7.85.252:39391/ using document root <none> and password file <none>
I20260812 00:11:31.342170 18437 fs_manager.cc:359] Metadata directory not provided
I20260812 00:11:31.342245 18437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:11:31.342410 18437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:11:31.343528 18437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data/instance:
uuid: "8b5c487efcf548539f06aedcc0038e10"
format_stamp: "Formatted at 2026-08-12 00:11:31 on dist-test-slave-z8x9"
server_key: "638c9c9a428bf88c3f6e0a553b8aeb1d"
server_key_iv: "b4466389a4116d8fba4e7d43e7f40dcc"
server_key_version: "encryptionkey@0"
I20260812 00:11:31.343924 18437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal/instance:
uuid: "8b5c487efcf548539f06aedcc0038e10"
format_stamp: "Formatted at 2026-08-12 00:11:31 on dist-test-slave-z8x9"
server_key: "638c9c9a428bf88c3f6e0a553b8aeb1d"
server_key_iv: "b4466389a4116d8fba4e7d43e7f40dcc"
server_key_version: "encryptionkey@0"
I20260812 00:11:31.345436 18437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 00:11:31.346304 18464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:11:31.346524 18437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 00:11:31.346633 18437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data,/tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal
uuid: "8b5c487efcf548539f06aedcc0038e10"
format_stamp: "Formatted at 2026-08-12 00:11:31 on dist-test-slave-z8x9"
server_key: "638c9c9a428bf88c3f6e0a553b8aeb1d"
server_key_iv: "b4466389a4116d8fba4e7d43e7f40dcc"
server_key_version: "encryptionkey@0"
I20260812 00:11:31.346738 18437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:11:31.362883 18282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } attempt: 2
I20260812 00:11:31.363469 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.363324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57272)
0812 00:11:31.363394 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.363438 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 00:11:31.363996 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.363848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57286)
0812 00:11:31.363928 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.363967 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 00:11:31.364210 18282 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.85.252:46359: Network error: Client connection negotiation failed: client connection to 127.7.85.252:46359: connect: Connection refused (error 111)
I20260812 00:11:31.402083 18437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:11:31.404633 18437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:11:31.404819 18437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:11:31.409466 18437 rpc_server.cc:307] RPC server started. Bound to: 127.7.85.252:46359
I20260812 00:11:31.409543 18569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.85.252:46359 every 8 connection(s)
I20260812 00:11:31.409934 18437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/data/info.pb
I20260812 00:11:31.411024 18571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:11:31.412629 18571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.413388 7511 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskaxXthI/build/release/bin/kudu as pid 18437
I20260812 00:11:31.413513 7511 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskaxXthI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786493185917310-7511-0/minicluster-data/master-2/wal/instance
I20260812 00:11:31.413749 7511 external_mini_cluster.cc:1428] Setting key 49a6b6b068a1d2a61544207f11a0c137
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:11:31.423377 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.413586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.7.85.254:36965)
0812 00:11:31.413649 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.413652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.413885 (+ 233us) server_negotiation.cc:407] Connection header received
0812 00:11:31.414512 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.414514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.414525 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.414591 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.415121 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.415745 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.417022 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.417360 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.417442 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.420182 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.420184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.420184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.420192 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.421703 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.422266 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.422267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.422267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.422324 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.422611 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.422612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.422612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.422737 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.422792 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.423152 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.423261 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:11:31.423466 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.413503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.254:36965 (local address 127.0.0.1:48396)
0812 00:11:31.413715 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.413766 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.414430 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.414603 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.414605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.414869 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.415027 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.415031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.415842 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.415843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.416932 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.416935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.417467 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.417468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.417571 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.418101 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.418112 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.420076 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.421791 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.421800 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.421802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.422151 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.422395 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.422396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.422396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.422536 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.422809 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.422816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.422988 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.423113 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.423256 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260812 00:11:31.424084 18571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.424521 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.414211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57292 (local address 127.7.85.252:46359)
0812 00:11:31.414356 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.414361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.414402 (+ 41us) server_negotiation.cc:407] Connection header received
0812 00:11:31.414491 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.414500 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.414533 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.414614 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.415497 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.416723 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.417763 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.418199 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.418277 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.421104 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.421114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.421114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.421121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.423044 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.423443 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.423446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.423447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.423509 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.423740 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.423741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.423742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.423925 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.424077 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.424308 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.424396 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.254@KRBTEST.COM
I20260812 00:11:31.432075 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.424401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.253:44689 (local address 127.0.0.1:59322)
0812 00:11:31.424443 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.424453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.424485 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.424829 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.424831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.424979 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.425030 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.425032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.425914 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.425915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.426588 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.426591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.427066 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.427067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.427185 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.427573 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.427578 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.429093 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.430889 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.430890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.430891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.431134 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.431386 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.431387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.431387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.431449 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.431764 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.431765 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.431828 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.431917 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.432006 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 00:11:31.432190 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.424493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59322 (local address 127.7.85.253:44689)
0812 00:11:31.424543 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.424545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.424553 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:11:31.424579 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.424580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.424591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.424824 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.425116 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.425824 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.426665 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.426976 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.427037 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.429230 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.429232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.429232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.429240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.430788 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.431229 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.431230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.431231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.431300 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.431522 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.431523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.431524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.431662 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.431716 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.431995 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.432079 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 00:11:31.432276 18423 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } attempt: 3
I20260812 00:11:31.432801 18571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:11:31.434053 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.425592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.7.85.254:36965)
0812 00:11:31.425628 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.425630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.425638 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:11:31.425693 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.425694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.425703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.425766 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.426321 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.426857 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.427813 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.428004 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.428066 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.430592 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.430594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.430594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.430603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.432724 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.433148 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.433149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.433149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.433256 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.433499 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.433499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.433500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.433617 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.433784 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.433861 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.433940 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.253@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493491, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.252@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.85.253@KRBTEST.COM
I20260812 00:11:31.441638 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.432610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57302 (local address 127.7.85.252:46359)
0812 00:11:31.432662 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.432664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.432673 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:31.432703 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.432704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.432713 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.432780 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.433118 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.433895 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.434851 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.435168 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.435228 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.437569 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.437571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.437571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.437578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.439911 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440296 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.440297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.440298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.440350 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440893 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.440893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.440894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.441148 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.441274 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.441455 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.441540 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:11:31.441685 18429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.432538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57302)
0812 00:11:31.432617 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.432627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.432660 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.432801 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.432802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.433000 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.433039 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.433040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.433985 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.433987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.434761 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.434763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.435247 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.435248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.435327 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.435731 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.435736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.437494 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.439997 (+ 2503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.439998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.439999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.440189 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.440432 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.440433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.440785 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.441276 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.441277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.441334 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.441425 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.441584 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 00:11:31.442145 18573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.433053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57306)
0812 00:11:31.433343 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.433353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.433398 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.433566 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.433567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.433709 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.433736 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.433737 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.434632 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.434633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.435344 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.435345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.435786 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.435787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.435845 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.436271 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.436276 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.437826 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.440388 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.440389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.440562 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.440766 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.440767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.440829 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.441839 (+ 1010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.441840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.441886 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.441957 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.442053 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275}
I20260812 00:11:31.442571 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.433116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57306 (local address 127.7.85.252:46359)
0812 00:11:31.433349 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.433351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.433386 (+ 35us) server_negotiation.cc:407] Connection header received
0812 00:11:31.433445 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.433446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.433456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.433525 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.433799 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.434552 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.435422 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.435712 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.435785 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.437913 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.437914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.437915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.437922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.440280 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440634 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.440635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.440636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.440689 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.440909 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.440910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.440910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.441136 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.441256 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.442333 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.442412 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":160,"threads_started":1}
I20260812 00:11:31.443182 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.434747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.7.85.253:44689)
0812 00:11:31.434781 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.434783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.434791 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:11:31.434853 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.434854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.434863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.434918 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.435455 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.436105 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.437098 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.437375 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.437459 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.440246 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.440247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.440248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.440255 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.442042 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.442425 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.442426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.442427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.442482 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.442709 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.442710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.442710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.442831 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.442974 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.443014 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.443089 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 00:11:31.443778 18423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: Bootstrap starting.
I20260812 00:11:31.444574 18423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:31.444666 7511 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:11:31.444890 18423 log.cc:824] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:31.445856 18423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: No bootstrap required, opened a new log
I20260812 00:11:31.446552 18571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: Bootstrap starting.
I20260812 00:11:31.445884 18235 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3e77e1fe51974235984011422032020b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48412
I20260812 00:11:31.447286 18571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:31.447602 18571 log.cc:824] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:31.448083 18423 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.448258 18423 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fed0fa93844b148148285a7b4ef3b0, State: Initialized, Role: FOLLOWER
I20260812 00:11:31.448369 18423 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.448513 18571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: No bootstrap required, opened a new log
I20260812 00:11:31.448729 18423 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:31.448689 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.448807 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:11:31.450670 18602 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:31.450730 18602 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:31.451072 18571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.451220 18571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b5c487efcf548539f06aedcc0038e10, State: Initialized, Role: FOLLOWER
I20260812 00:11:31.451311 18571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.451617 18571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:31.451607 18608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.451718 18608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:11:31.453505 18629 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:31.453604 18629 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:31.451047 18293 master_runner.cc:186] Error getting master registration for 127.7.85.254:36965: OK, instance_id { permanent_uuid: "3e77e1fe51974235984011422032020b" instance_seqno: 1786493491011492 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:11:31.455641 18293 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:11:31.453768 18437 master_runner.cc:186] Error getting master registration for 127.7.85.254:36965: OK, instance_id { permanent_uuid: "3e77e1fe51974235984011422032020b" instance_seqno: 1786493491011492 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:11:31.456876 18437 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:11:31.463374 18282 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } attempt: 3
Aug 12 00:11:31 dist-test-slave-z8x9 krb5kdc[18129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786493490, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.85.254@KRBTEST.COM for kudu/127.7.85.252@KRBTEST.COM
I20260812 00:11:31.472537 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.463718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.85.252:46359 (local address 127.0.0.1:57318)
0812 00:11:31.463778 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 00:11:31.463794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:11:31.463860 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:11:31.464056 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:11:31.464057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:11:31.464270 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:11:31.464337 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.464339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.465350 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.465351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.466164 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:11:31.466167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.466706 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.466707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:11:31.466763 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.467307 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:11:31.467312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:11:31.469079 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:11:31.471245 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.471246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.471247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.471502 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.471725 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:11:31.471726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:11:31.471726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:11:31.471822 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:11:31.472184 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:11:31.472185 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:11:31.472235 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 00:11:31.472307 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 00:11:31.472421 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 00:11:31.472558 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.463877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.7.85.252:46359)
0812 00:11:31.463921 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.463923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.463932 (+ 9us) server_negotiation.cc:407] Connection header received
0812 00:11:31.463960 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.463961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.463974 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.464036 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.464427 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.465264 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.466269 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.466630 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.466705 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.469226 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.469228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.469228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.469237 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.471136 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.471581 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.471583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.471583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.471648 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.471933 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.471934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.471935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.472090 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.472156 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.472360 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.472461 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:11:31.474747 18282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: Bootstrap starting.
I20260812 00:11:31.475574 18282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: Neither blocks nor log segments found. Creating new log.
I20260812 00:11:31.475876 18282 log.cc:824] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:11:31.476807 18282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: No bootstrap required, opened a new log
I20260812 00:11:31.478439 18282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.478602 18282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e77e1fe51974235984011422032020b, State: Initialized, Role: FOLLOWER
I20260812 00:11:31.478694 18282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.479061 18235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 00:11:31.479158 18235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:31.479075 18634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.479393 18634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:31.480084 18235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.480369 18235 leader_election.cc:288] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [CANDIDATE]: Term 1 election: Requested vote from peers a4fed0fa93844b148148285a7b4ef3b0 (127.7.85.253:44689), 8b5c487efcf548539f06aedcc0038e10 (127.7.85.252:46359)
I20260812 00:11:31.479045 18282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:11:31.480959 18376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e77e1fe51974235984011422032020b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a4fed0fa93844b148148285a7b4ef3b0"
I20260812 00:11:31.481091 18376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:31.482064 18376 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e77e1fe51974235984011422032020b in term 1.
W20260812 00:11:31.483108 18661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:11:31.483160 18661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:11:31.483088 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e77e1fe51974235984011422032020b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8b5c487efcf548539f06aedcc0038e10"
I20260812 00:11:31.483212 18517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:11:31.485872 18180 leader_election.cc:302] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e77e1fe51974235984011422032020b, a4fed0fa93844b148148285a7b4ef3b0; no voters:
I20260812 00:11:31.486889 18517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e77e1fe51974235984011422032020b in term 1.
I20260812 00:11:31.488142 18147 master_runner.cc:263] No leader master found from master 3e77e1fe51974235984011422032020b
I20260812 00:11:31.488189 18147 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:11:31.487323 18634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:11:31.488453 18634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [term 1 LEADER]: Becoming Leader. State: Replica: 3e77e1fe51974235984011422032020b, State: Running, Role: LEADER
I20260812 00:11:31.488597 18634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } }
I20260812 00:11:31.489130 18634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e77e1fe51974235984011422032020b. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.489297 18634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:31.497323 18675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:11:31.498239 18675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:11:31.505635 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.489682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.7.85.254:36965)
0812 00:11:31.489729 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.489732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.490305 (+ 573us) server_negotiation.cc:407] Connection header received
0812 00:11:31.490423 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.490424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.490435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.490465 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.491324 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.492195 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.493303 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.493612 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.493656 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.495553 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.495554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.495555 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.495562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.497309 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.503562 (+ 6253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.503564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.503564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.503648 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.504943 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.504944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.504945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.505101 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.505206 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.505435 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.505525 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 00:11:31.508468 18191 tablet.cc:2369] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:11:31.509895 18517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [term 1 FOLLOWER]: Refusing update from remote peer 3e77e1fe51974235984011422032020b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:11:31.510272 18671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:11:31.511801 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.489439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59336 (local address 127.7.85.253:44689)
0812 00:11:31.489482 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.489485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.497414 (+ 7929us) server_negotiation.cc:407] Connection header received
0812 00:11:31.497530 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.497532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.497544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.497575 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.498614 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.499320 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.502643 (+ 3323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.502957 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.503002 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.508285 (+ 5283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.508288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.508288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.508297 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.510158 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.510828 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.510830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.510831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.510897 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.511111 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.511112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.511112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.511266 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.511366 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.511608 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.511697 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:11:31.511900 18608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e77e1fe51974235984011422032020b. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.512010 18608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:31.512501 18376 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [term 1 FOLLOWER]: Refusing update from remote peer 3e77e1fe51974235984011422032020b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:11:31.512840 18671 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:11:31.513399 18671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.513500 18671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:31.514014 18608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.514101 18608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:31.514364 18684 mvcc.cc:205] Tried to move back new op lower bound from 7317477341221044224 to 7317477341138624512. Current Snapshot: MvccSnapshot[applied={T|T < 7317477341221044224}]
I20260812 00:11:31.514982 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e77e1fe51974235984011422032020b. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.515069 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:31.519850 18577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:31.489974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.7.85.252:46359)
0812 00:11:31.490013 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.490014 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.497854 (+ 7840us) server_negotiation.cc:407] Connection header received
0812 00:11:31.497951 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.497952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.497963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.497993 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.500916 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.501638 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.506347 (+ 4709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:31.506672 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:31.506719 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:31.515854 (+ 9135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:11:31.515858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:11:31.515859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:11:31.515875 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:11:31.518466 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.518940 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.518942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.518942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.519005 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:11:31.519217 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:11:31.519218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:11:31.519219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:11:31.519363 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:11:31.519463 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:31.519665 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 00:11:31.519752 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 00:11:31.521713 18675 catalog_manager.cc:1347] Generated new cluster ID: e89c3b5c965e4594bb4ad984b677cf15
I20260812 00:11:31.521809 18675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:11:31.527281 18634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.527407 18634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b [sys.catalog]: This master's current role is: LEADER
I20260812 00:11:31.527932 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e77e1fe51974235984011422032020b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e77e1fe51974235984011422032020b" member_type: VOTER last_known_addr { host: "127.7.85.254" port: 36965 } } peers { permanent_uuid: "a4fed0fa93844b148148285a7b4ef3b0" member_type: VOTER last_known_addr { host: "127.7.85.253" port: 44689 } } peers { permanent_uuid: "8b5c487efcf548539f06aedcc0038e10" member_type: VOTER last_known_addr { host: "127.7.85.252" port: 46359 } } }
I20260812 00:11:31.528034 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:11:31.533255 18692 mvcc.cc:205] Tried to move back new op lower bound from 7317477341221044224 to 7317477341138624512. Current Snapshot: MvccSnapshot[applied={T|T < 7317477341221044224}]
I20260812 00:11:31.563745 18675 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:11:31.564531 18675 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:11:31.570996 18675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e77e1fe51974235984011422032020b: Generated new TSK 0
I20260812 00:11:31.571231 18675 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:11:31.604341 18285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.602862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.7.85.254:36965)
0812 00:11:31.602912 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.602914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.603587 (+ 673us) server_negotiation.cc:407] Connection header received
0812 00:11:31.603703 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.603704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.603716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.603749 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.604258 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48428: BlockingRecv error: recv got EOF from 127.0.0.1:48428 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 00:11:31.605584 18577 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.603255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.7.85.252:46359)
0812 00:11:31.603304 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.603306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.604994 (+ 1688us) server_negotiation.cc:407] Connection header received
0812 00:11:31.605097 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.605098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.605109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.605141 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.605521 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57334: BlockingRecv error: recv got EOF from 127.0.0.1:57334 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 00:11:31.606402 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.604818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.7.85.253:44689)
0812 00:11:31.604877 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.604879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.605820 (+ 941us) server_negotiation.cc:407] Connection header received
0812 00:11:31.605929 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.605930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.605941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.605972 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.606338 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59338: BlockingRecv error: recv got EOF from 127.0.0.1:59338 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 00:11:31.614706 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.609481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.7.85.253:44689)
0812 00:11:31.609524 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.609526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.614030 (+ 4504us) server_negotiation.cc:407] Connection header received
0812 00:11:31.614165 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.614166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.614191 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.614224 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.614628 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59348: BlockingRecv error: recv got EOF from 127.0.0.1:59348 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 00:11:31.618386 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.617513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59358 (local address 127.7.85.253:44689)
0812 00:11:31.617564 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.617566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.617770 (+ 204us) server_negotiation.cc:407] Connection header received
0812 00:11:31.617880 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.617881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.617893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.617925 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.618325 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59358: BlockingRecv error: recv got EOF from 127.0.0.1:59358 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 00:11:31.622785 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.621866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59368 (local address 127.7.85.253:44689)
0812 00:11:31.621909 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.621911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.622176 (+ 265us) server_negotiation.cc:407] Connection header received
0812 00:11:31.622288 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.622290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.622302 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.622332 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.622724 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59368: BlockingRecv error: recv got EOF from 127.0.0.1:59368 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 00:11:31.629021 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.628128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59376 (local address 127.7.85.253:44689)
0812 00:11:31.628168 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.628170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.628413 (+ 243us) server_negotiation.cc:407] Connection header received
0812 00:11:31.628519 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.628520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.628532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.628565 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.628962 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59376: BlockingRecv error: recv got EOF from 127.0.0.1:59376 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 00:11:31.639467 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.638512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.7.85.253:44689)
0812 00:11:31.638562 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.638565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.638849 (+ 284us) server_negotiation.cc:407] Connection header received
0812 00:11:31.638956 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.638957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.638969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.638999 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.639400 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59382: BlockingRecv error: recv got EOF from 127.0.0.1:59382 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 00:11:31.657991 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.657098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59398 (local address 127.7.85.253:44689)
0812 00:11:31.657223 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.657226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.657386 (+ 160us) server_negotiation.cc:407] Connection header received
0812 00:11:31.657494 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.657495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.657508 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.657537 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.657925 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59398: BlockingRecv error: recv got EOF from 127.0.0.1:59398 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260812 00:11:31.692830 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.691667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.7.85.253:44689)
0812 00:11:31.691787 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.691789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.691802 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:11:31.691853 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.691854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.691866 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.691936 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.692733 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59404: BlockingRecv error: recv got EOF from 127.0.0.1:59404 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 00:11:31.759181 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.758334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59420 (local address 127.7.85.253:44689)
0812 00:11:31.758431 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.758434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.758539 (+ 105us) server_negotiation.cc:407] Connection header received
0812 00:11:31.758638 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.758639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.758652 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.758730 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.759106 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59420: BlockingRecv error: recv got EOF from 127.0.0.1:59420 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 00:11:31.889405 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:31.888553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59422 (local address 127.7.85.253:44689)
0812 00:11:31.888662 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:31.888664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:31.888677 (+ 13us) server_negotiation.cc:407] Connection header received
0812 00:11:31.888749 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:31.888750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:31.888762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:31.888815 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:31.889324 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59422: BlockingRecv error: recv got EOF from 127.0.0.1:59422 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 00:11:32.147428 18431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:11:32.146779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.7.85.253:44689)
0812 00:11:32.146874 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:32.146876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:32.146887 (+ 11us) server_negotiation.cc:407] Connection header received
0812 00:11:32.146919 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:32.146920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:32.146940 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:32.147010 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:11:32.147378 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59432: BlockingRecv error: recv got EOF from 127.0.0.1:59432 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 00:11:32.451196 18602 catalog_manager.cc:1259] Loaded cluster ID: e89c3b5c965e4594bb4ad984b677cf15
I20260812 00:11:32.451289 18602 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: loading cluster ID for follower catalog manager: success
I20260812 00:11:32.452312 18602 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: acquiring CA information for follower catalog manager: success
I20260812 00:11:32.452582 18602 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a4fed0fa93844b148148285a7b4ef3b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:11:32.453990 18629 catalog_manager.cc:1259] Loaded cluster ID: e89c3b5c965e4594bb4ad984b677cf15
I20260812 00:11:32.454047 18629 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: loading cluster ID for follower catalog manager: success
I20260812 00:11:32.454816 18629 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: acquiring CA information for follower catalog manager: success
I20260812 00:11:32.455104 18629 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b5c487efcf548539f06aedcc0038e10: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:11:32.665115 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:32.660996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.7.85.253:44689)
0812 00:11:32.661115 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:32.661118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:32.661228 (+ 110us) server_negotiation.cc:407] Connection header received
0812 00:11:32.661333 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:32.661334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:32.661439 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:32.661548 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:32.661740 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.662706 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.664005 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.664351 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.664422 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:32.664633 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:32.664833 (+ 200us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:32.664931 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:32.664973 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 00:11:32.664999 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260812 00:11:32.670791 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:32.667314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.7.85.252:46359)
0812 00:11:32.667677 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:32.667679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:32.667687 (+ 8us) server_negotiation.cc:407] Connection header received
0812 00:11:32.667772 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:32.667773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:32.667838 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:32.667935 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:32.668042 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.668772 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.669887 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.670111 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.670211 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:32.670389 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:32.670515 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:32.670605 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:32.670635 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 00:11:32.670657 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":59,"threads_started":1}
I20260812 00:11:32.675793 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:32.672864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.7.85.253:44689)
0812 00:11:32.672933 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:32.672936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:32.673038 (+ 102us) server_negotiation.cc:407] Connection header received
0812 00:11:32.673139 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:32.673140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:32.673188 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:32.673251 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:32.673430 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.674223 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.675042 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.675262 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.675341 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:32.675505 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:32.675546 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:32.675603 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:32.675681 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 00:11:32.675705 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:11:32.676489 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:32.673211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.7.85.252:46359)
0812 00:11:32.673255 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:32.673258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:32.673352 (+ 94us) server_negotiation.cc:407] Connection header received
0812 00:11:32.673418 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:32.673419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:32.673431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:32.673483 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:32.673698 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.674377 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.675464 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.675783 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.675832 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:32.676219 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:32.676253 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:32.676308 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:32.676360 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 00:11:32.676397 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 00:11:32.676554 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:11:32.672835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.7.85.254:36965)
0812 00:11:32.673321 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 00:11:32.673323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 00:11:32.673330 (+ 7us) server_negotiation.cc:407] Connection header received
0812 00:11:32.673376 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:11:32.673377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:11:32.673455 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:11:32.673534 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:11:32.673718 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.674257 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.675454 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:11:32.675770 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:11:32.675829 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:11:32.676064 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:11:32.676197 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:11:32.676257 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:11:32.676314 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 00:11:32.676338 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":64,"threads_started":1}
I20260812 00:11:32.678498 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18147
I20260812 00:11:32.694222 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18293
I20260812 00:11:32.707645 7511 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskaxXthI/build/release/bin/kudu with pid 18437
2026-08-12T00:11:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3356 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6685 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (306796 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST